Chebyshev approaches for imbalanced data streams regression models

Abstract

In recent years data stream mining and learning from imbalanced data have been active research areas. Even though solutions exist to tackle these two problems, most of them are not designed to handle challenges inherited from both problems. As far as we are aware, the few approaches in the area of learning from imbalanced data streams fall in the context of classification, and no efforts on the regression domain have been reported yet. This paper proposes a technique that uses sampling strategies to cope with imbalanced data streams in a regression setting, where the most important cases have rare and extreme target values. Specifically, we employ under-sampling and over-sampling strategies that resort to Chebyshev’s inequality value as a heuristic to disclose the type of incoming cases (i.e. frequent or rare). We have evaluated our proposal by applying it in the training of models by four well-known regression algorithms over fourteen benchmark data sets. We conducted a series of experiments with different setups on both synthetic and real-world data sets. The experimental results confirm our approach’s effectiveness by showing the models’ superior performance trained by each of the sampling strategies compared with their baseline pairs.

Appendices

Experimental evaluation tables

1.1 More about the data sets

Since we have used short names to refer to the data sets in the main text and make it easier for the readers to find the data sets, we report their complete name, source, and the number of their attributes in Table 8.

Table 9 Data sets characteristics: number of cases (#cases), number of cases with rare extreme target values (#rare cases) with $thr_\phi =0.9$ that are low extremes (i.e below the median), high extremes (i.e. above the median) or both

Table 10 Data sets characteristics : number of cases (#cases), number of cases with rare extreme target values (#rare cases) with $thr_\phi =1.0$ that are low extremes (i.e below the median), high extremes (i.e. above the median) or both

1.2 Results of the experiments: sensitivity analysis varying $\phi$

Tables 11, 12, 13, 14, 15 and 16 show results of the paired comparisons for the four learner models trained by our (Under/Over)-sampling strategies, ChebyUS and ChebyOS, against their baseline versions. The symbols $\triangleright$ and $\triangleleft$ indicate that the ChebyUS or ChebyOS method is significantly better or worse, respectively, compared to the baseline. In each experiment, the value obtained by the error function (i.e. $RMSE_{\phi }$ or RMSE) for prediction of the learner mentioned in the column header over the data set specified in the corresponding row has been reported. The numbers inside the cells are the average and corresponding standard deviation of the results on ten rounds of experiments. We report the statistical significance (level 95%) of the difference between each pair using the two symbols $\triangleright$ and $\triangleleft$ pointing to the significantly better method. As we have reported results for different levels of $thr_\phi$, information about the number/percentage of the rare cases observed in each data set presented in Tables 9 and 10.

Table 11 RMSE and $RMSE_{\phi }$ estimates considering all the observations: $thr_\phi = 0$

Table 12 RMSE and $RMSE_{\phi }$ estimates considering all (double-side) rare cases: $thr_\phi = 0.8$

Table 13 RMSE and $RMSE_{\phi }$ estimates considering all (double-side) rare cases: $thr_\phi = 0.9$

Table 14 RMSE and $RMSE_{\phi }$ estimates considering all (double-side) rare cases: $thr_\phi = 1.0$

Chebyshev’s probability, K value and relevance function $\phi ()$ graphs for each data set

The value of function $\phi ()$ to each example of the data sets along with their given probability by Algorithm 1, their given K value by Algorithm 2 and also the box plot of the data set have been shown in the following figures. The red points on the graphs indicate the rare cases considered in each data set.

Table 15 $RMSE_{\phi }$ estimates considering left-hand side rare cases ($thr_\phi = 0.8$)

Table 16 $RMSE_{\phi }$ estimates considering right-hand side rare cases ($thr_\phi = 0.8$)

Fig. 15

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of puma32H data set

Fig. 16

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of cpusum data set

Fig. 17

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of elevator data set

Fig. 18

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of bike data set

Fig. 19

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of energy data set

Fig. 20

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of calhousing data set

Fig. 21

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of gasemission data set

Fig. 22

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of mv data set

Fig. 23

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of fried data set

Fig. 24

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of pollution data set

Fig. 25

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of car price data set

Fig. 26

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of query data set

Fig. 27

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of GPU data set

Fig. 28

Box plot, K-value, Chebyshev’s probability and relevance function $\phi ()$ for the target variable of 3d spatial network data set