Paired feature multilayer ensemble – concept and evaluation of a classifier

Abstract

Recently, the representation learning is the fucus of intense research of machine learning community. The underlying idea is that the key for successful discrimination of difficult datasets is a good feature extraction. A transformation of the data space into another space where classification is easy. This work proposes a novel transformation into feature space that follows a photographic intuition: that we can build from pairs of features in original space some kind of photographic plate where the sample data are projected to create a picture of the data distribution in the feature subspace defined by the feature pair. These photographic plates may be used as individuals of a classifier ensemble. The approach allows a natural definition of a confidence weight affecting each individual classifier out for the construction of a combination rule used by the ensemble. Hence the name Paired Feature Multilayer Ensemble (PFME). The approach is naturally naive parallel, insensitive to sample size, robust to dimension increase, and allows a regularization in feature space which is independent from original input space. The proposed approach was evaluated on the basis of the computer experiments carried out on the benchmark datasets.