B

Deep Variational Implicit Processes Full Regression Results

Table B.1: Experiment results on Deep Variational Implicit Processes (Chapter 3). Log Likelihood, Root Mean Squared Error and Continuous Ranked Probability Score results on regression UCI benchmark datasets. Eight datasets are considered: Boston, Energy, Concrete, WineRed, Power, Naval, Protein, and Kin8nm. Results are averaged over 20 random train-test splits. Best value is highlighted in purple and second-to-best in teal.
NLL Single-layer DVIP (Salimbeni & Deisenroth, 2017)
SGP VIP VIP 200 DVIP 2 DVIP 3 DVIP 4 DVIP 5 DGP 2 DGP 3 DGP 4 DGP 5
Boston \(\bm{2.62 \pm 0.05}\) \(2.76 \pm 0.05\) \(2.69 \pm 0.03\) \(2.85 \pm 0.09\) \(\bm{2.59 \pm 0.06}\) \(2.67 \pm 0.09\) \(2.66 \pm 0.08\) \(2.63 \pm 0.05\) \(2.63 \pm 0.05\) \(2.64 \pm 0.05\) \(2.65 \pm 0.05\)
Energy \(1.54 \pm 0.02\) \(2.07 \pm 0.02\) \(2.07 \pm 0.02\) \(0.76 \pm 0.02\) \(\bm{0.70 \pm 0.01}\) \(\bm{0.70 \pm 0.01}\) \(0.73 \pm 0.01\) \(\bm{0.72 \pm 0.01}\) \(0.74 \pm 0.01\) \(\bm{0.72 \pm 0.01}\) \(0.73 \pm 0.01\)
Concrete \(3.16 \pm 0.01\) \(3.45 \pm 0.02\) \(3.48 \pm 0.01\) \(3.24 \pm 0.04\) \(3.20 \pm 0.05\) \(\bm{3.03 \pm 0.02}\) \(\bm{3.06 \pm 0.02}\) \(3.17 \pm 0.01\) \(3.20 \pm 0.01\) \(3.13 \pm 0.01\) \(3.12 \pm 0.01\)
Winered \(\bm{0.93 \pm 0.01}\) \(\bm{0.94 \pm 0.01}\) \(0.96 \pm 0.01\) \(\bm{0.94 \pm 0.01}\) \(\bm{0.94 \pm 0.01}\) \(\bm{0.94 \pm 0.01}\) \(0.95 \pm 0.01\) \(\bm{0.94 \pm 0.01}\) \(\bm{0.94 \pm 0.01}\) \(\bm{0.94 \pm 0.01}\) \(\bm{0.93 \pm 0.01}\)
Power \(2.84 \pm 0.00\) \(2.85 \pm 0.00\) \(2.86 \pm 0.00\) \(2.82 \pm 0.01\) \(2.81 \pm 0.00\) \(\bm{2.79 \pm 0.01}\) \(\bm{2.79 \pm 0.01}\) \(2.81 \pm 0.01\) \(\bm{2.80 \pm 0.00}\) \(\bm{2.80 \pm 0.00}\) \(\bm{2.80 \pm 0.01}\)
Protein \(2.93 \pm 0.00\) \(3.03 \pm 0.00\) \(3.03 \pm 0.00\) \(2.93 \pm 0.00\) \(2.89 \pm 0.00\) \(2.88 \pm 0.00\) \(2.86 \pm 0.00\) \(2.84 \pm 0.00\) \(\bm{2.79 \pm 0.00}\) \(\bm{2.79 \pm 0.00}\) \(\bm{2.80 \pm 0.00}\)
Naval \(-6.11 \pm 0.06\) \(-4.50 \pm 0.02\) \(-4.31 \pm 0.00\) \(-5.89 \pm 0.02\) \(-5.98 \pm 0.01\) \(-5.90 \pm 0.01\) \(-5.92 \pm 0.01\) \(\bm{-6.35 \pm 0.09}\) \(-6.21 \pm 0.04\) \(\bm{-6.27 \pm 0.06}\) \(-6.21 \pm 0.08\)
Kin8nm \(-0.91 \pm 0.00\) \(-0.31 \pm 0.00\) \(-0.25 \pm 0.00\) \(-1.00 \pm 0.00\) \(-1.13 \pm 0.00\) \(-1.15 \pm 0.00\) \(-1.16 \pm 0.00\) \(-1.29 \pm 0.00\) \(\bm{-1.32 \pm 0.00}\) \(\bm{-1.33 \pm 0.00}\) \(1.30 \pm 0.00\)
RMSE Single-layer DVIP (Salimbeni & Deisenroth, 2017)
SGP VIP VIP 200 DVIP 2 DVIP 3 DVIP 4 DVIP 5 DGP 2 DGP 3 DGP 4 DGP 5
Boston \(\bm{3.48 \pm 0.17}\) \(4.78 \pm 0.28\) \(4.49 \pm 0.28\) \(3.87 \pm 0.19\) \(\bm{3.50 \pm 0.20}\) \(3.60 \pm 0.19\) \(3.66 \pm 0.21\) \(3.51 \pm 0.18\) \(3.53 \pm 0.19\) \(3.55 \pm 0.20\) \(3.56 \pm 0.20\)
Energy \(1.07 \pm 0.03\) \(2.57 \pm 0.08\) \(2.68 \pm 0.07\) \(0.52 \pm 0.01\) \(\bm{0.47 \pm 0.01}\) \(\bm{0.46 \pm 0.01}\) \(\bm{0.47 \pm 0.01}\) \(\bm{0.46 \pm 0.01}\) \(\bm{0.47 \pm 0.01}\) \(\bm{0.46 \pm 0.01}\) \(\bm{0.46 \pm 0.01}\)
Concrete \(5.84 \pm 0.12\) \(7.75 \pm 0.15\) \(8.06 \pm 0.16\) \(6.01 \pm 0.16\) \(5.68 \pm 0.18\) \(\bm{5.13 \pm 0.12}\) \(\bm{5.27 \pm 0.13}\) \(5.86 \pm 0.12\) \(6.01 \pm 0.12\) \(5.54 \pm 0.11\) \(5.52 \pm 0.12\)
Winered \(\bm{0.61 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(0.63 \pm 0.00\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\) \(\bm{0.62 \pm 0.00}\)
Power \(4.15 \pm 0.03\) \(4.21 \pm 0.03\) \(4.22 \pm 0.03\) \(4.06 \pm 0.04\) \(4.01 \pm 0.04\) \(3.97 \pm 0.04\) \(\bm{3.95 \pm 0.04}\) \(4.00 \pm 0.04\) \(3.98 \pm 0.03\) \(3.99 \pm 0.03\) \(\bm{3.96 \pm 0.04}\)
Protein \(4.56 \pm 0.01\) \(5.05 \pm 0.01\) \(5.04 \pm 0.01\) \(4.54 \pm 0.01\) \(4.40 \pm 0.01\) \(4.33 \pm 0.01\) \(4.26 \pm 0.01\) \(4.17 \pm 0.01\) \(\bm{4.00 \pm 0.01}\) \(\bm{4.01 \pm 0.01}\) \(4.02 \pm 0.01\)
Naval \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\)
Kin8nm \(0.09 \pm 0.00\) \(0.17 \pm 0.00\) \(0.18 \pm 0.00\) \(0.08 \pm 0.00\) \(\bm{0.07 \pm 0.00}\) \(\bm{0.07 \pm 0.00}\) \(\bm{0.07 \pm 0.00}\) \(\bm{0.06 \pm 0.00}\) \(\bm{0.06 \pm 0.00}\) \(\bm{0.06 \pm 0.00}\) \(\bm{0.06 \pm 0.00}\)
CRPS Single-layer DVIP (Salimbeni & Deisenroth, 2017)
SGP VIP VIP 200 DVIP 2 DVIP 3 DVIP 4 DVIP 5 DGP 2 DGP 3 DGP 4 DGP 5
Boston \(1.79 \pm 0.05\) \(2.25 \pm 0.08\) \(2.13 \pm 0.08\) \(1.91 \pm .06\) \(\bm{1.76 \pm 0.07}\) \(1.81 \pm 0.07\) \(\bm{1.78 \pm 0.06}\) \(1.79 \pm 0.05\) \(1.80 \pm 0.06\) \(1.80 \pm 0.06\) \(1.81 \pm 0.06\)
Energy \(0.62 \pm 0.01\) \(1.27 \pm 0.04\) \(1.30 \pm 0.03\) \(\bm{0.28 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\) \(\bm{0.26 \pm 0.00}\)
Concrete \(3.20 \pm 0.05\) \(4.29 \pm 0.08\) \(4.43 \pm 0.08\) \(3.26 \pm 0.07\) \(3.03 \pm 0.09\) \(\bm{2.74 \pm 0.05}\) \(\bm{2.83 \pm 0.05}\) \(3.21 \pm 0.05\) \(3.31 \pm 0.05\) \(3.05 \pm 0.05\) \(3.04 \pm 0.05\)
Winered \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.35 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\) \(\bm{0.34 \pm 0.00}\)
Power \(2.27 \pm 0.01\) \(2.31 \pm 0.01\) \(2.31 \pm 0.01\) \(2.21 \pm 0.01\) \(2.18 \pm 0.01\) \(\bm{2.14 \pm 0.01}\) \(\bm{2.14 \pm 0.01}\) \(2.17 \pm 0.01\) \(2.16 \pm 0.01\) \(2.17 \pm 0.01\) \(\bm{2.15 \pm 0.01}\)
Protein \(2.56 \pm 0.00\) \(2.87 \pm 0.00\) \(2.86 \pm 0.01\) \(2.54 \pm 0.00\) \(2.43 \pm 0.00\) \(2.38 \pm 0.00\) \(2.33 \pm 0.00\) \(2.31 \pm 0.00\) \(\bm{2.19 \pm 0.00}\) \(\bm{2.19 \pm 0.00}\) \(\bm{2.20 \pm 0.00}\)
Naval \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\) \(\bm{0.00 \pm 0.00}\)
Kin8nm \(0.05 \pm 0.00\) \(0.09 \pm 0.0\) \(0.10 \pm 0.00\) \(\bm{0.04 \pm 0.00}\) \(\bm{0.04 \pm 0.00}\) \(\bm{0.04 \pm 0.00}\) \(\bm{0.04 \pm 0.00}\) \(\bm{0.03 \pm 0.00}\) \(\bm{0.03 \pm 0.00}\) \(\bm{0.03 \pm 0.00}\) \(\bm{0.03 \pm 0.00}\)