Accuracy bounds for ensembles under 0 - 1 loss.

Abstract

This paper is an attempt to increase the understanding in the behavior of ensembles for discrete variables in a quantitative way. A set of tight upper and lower bounds for the accuracy of an ensemble is presented for wide classes of ensemble algorithms, including bagging and boosting. The ensemble accuracy is expressed in terms of the accuracies of the members of the ensemble. Since those bounds represent best and worst case behavior only, we study typical behavior as well, and discuss its properties. A parameterised bound is presented which describes ensemble bahavior as a mixture of dependent base classifier and independent base classifier areas. Some empirical results are presented to support our conclusions.