Published in: Pattern Recognition, 34, (2), 2001, 299–314.
Decision Templates for Multiple Classifier Fusion: An Experimental Comparison Ludmila I. Kuncheva School of Mathematics, University of Wales, Bangor Bangor, Gwynedd LL57 1UT, United Kingdom, (
[email protected]) James C. Bezdek∗ Department of Computer Science, University of West Florida Pensacola, FL 32514, USA, (
[email protected]) Robert P.W. Duin Faculty of Applied Sciences, Delft University of Technology P.O. Box 5046, 2600 GA Delft, The Netherlands, (
[email protected])
Abstract Multiple classifier fusion may generate more accurate classification than each of the constituent classifiers. Fusion is often based on fixed combination rules like the product and average. Only under strict probabilistic conditions can these rules be justified. We present here a simple rule for adapting the class combiner to the application. c decision templates (one per class) are estimated with the same training set that is used for the set of classifiers. These templates are then matched to the decision profile of new incoming objects by some similarity measure. We compare 11 versions of our model with 14 other techniques for classifier fusion on the Satimage and Phoneme datasets from the database ELENA. Our results show that decision templates based on integral type measures of similarity are superior to the other schemes on both data sets. Keywords: Classifier fusion, Combination of multiple classifiers, Decision templates, Fuzzy similarity, Behavior-Knowledge-Space, Fuzzy integral, Dempster-Shafer, Class-conscious fusion, Class-indifferent fusion.
1
Introduction
Combining classifiers to achieve higher accuracy is an important research topic with different names in the literature: • combination of multiple classifiers ([1, 2, 3, 4, 5]); • classifier fusion ([6, 7, 8, 9, 10]); • mixture of experts ([11, 12, 13, 14]); • committees of neural networks ([15, 16]); • consensus aggregation ([17, 18, 19]); • voting pool of classifiers ([20]); • dynamic classifier selection ([3]); ∗ Research
supported by ONR grant N00014-96-1-0642
1
• composite classifier system ([21]); • classifier ensembles ([16, 22]), • divide-and-conquer classifiers [23]; • pandemonium system of reflective agents [24]; • change-glasses approach to classifier selection [25], etc. The paradigms of these models differ on the: assumptions about classifier dependencies; type of classifier outputs; aggregation strategy (global or local); aggregation procedure (a function, a neural network, an algorithm), etc. There are generally two types of combination: classifier selection and classifier fusion ([3]). The presumption in classifier selection is that each classifier is “an expert” in some local area of the feature space. When a feature vector x ∈
