File size: 2,485 Bytes
929e74d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
K=1  median regret (%) per draw [123 | 321]
  retr orig              3.83 |   3.06
  retr multi-A rows      0.56 |   0.26
  retr multi-B rows      0.21 |   0.13
  retr multi-B top1      0.21 |   0.13
  shipped                3.32 |   3.20
  tuned B                0.43 |   0.34
  tuned A                0.70 |   0.51
  tuned B minus retr orig          123: -3.41 p=2.5e-11 | 321: -2.74 p=1.8e-12
  tuned B minus retr multi-A rows  123: -0.07 p=0.58 | 321: +0.15 p=0.1
  tuned B minus retr multi-B rows  123: +0.12 p=0.11 | 321: +0.19 p=0.00094
  tuned B minus retr multi-B top1  123: +0.12 p=0.11 | 321: +0.19 p=0.00094
K=2  median regret (%) per draw [123 | 321]
  retr orig              2.86 |   2.54
  retr multi-A rows      0.20 |   0.13
  retr multi-B rows      0.28 |   0.13
  retr multi-B top1      0.15 |   0.08
  shipped                3.10 |   3.05
  tuned B                0.27 |   0.31
  tuned A                0.48 |   0.55
  tuned B minus retr orig          123: -2.33 p=9.1e-12 | 321: -2.25 p=9.1e-12
  tuned B minus retr multi-A rows  123: +0.04 p=0.54 | 321: +0.17 p=0.14
  tuned B minus retr multi-B rows  123: -0.06 p=0.29 | 321: +0.13 p=0.075
  tuned B minus retr multi-B top1  123: +0.11 p=0.064 | 321: +0.18 p=0.0041
K=4  median regret (%) per draw [123 | 321]
  retr orig              1.43 |   1.59
  retr multi-A rows      0.16 |   0.13
  retr multi-B rows      0.32 |   0.13
  retr multi-B top1      0.15 |   0.13
  shipped                2.95 |   2.90
  tuned B                0.28 |   0.40
  tuned A                0.37 |   0.51
  tuned B minus retr orig          123: -1.20 p=7.8e-09 | 321: -1.26 p=3.1e-10
  tuned B minus retr multi-A rows  123: +0.08 p=0.023 | 321: +0.24 p=0.00028
  tuned B minus retr multi-B rows  123: -0.12 p=0.045 | 321: +0.17 p=0.038
  tuned B minus retr multi-B top1  123: +0.09 p=0.036 | 321: +0.23 p=0.0056
K=8  median regret (%) per draw [123 | 321]
  retr orig              0.92 |   0.92
  retr multi-A rows      0.14 |   0.10
  retr multi-B rows      0.22 |   0.15
  retr multi-B top1      0.14 |   0.15
  shipped                2.83 |   2.74
  tuned B                0.24 |   0.37
  tuned A                0.51 |   0.53
  tuned B minus retr orig          123: -0.64 p=1.4e-08 | 321: -0.50 p=1.7e-06
  tuned B minus retr multi-A rows  123: +0.16 p=3.5e-05 | 321: +0.33 p=4.2e-06
  tuned B minus retr multi-B rows  123: +0.01 p=0.27 | 321: +0.14 p=0.1
  tuned B minus retr multi-B top1  123: +0.12 p=0.014 | 321: +0.25 p=0.022