Voting with random neural networks: A democratic ensemble classifier

Abstract

One of the main problems in machine learning is the investigation of the relationships between features in different datasets. In this case, using different classifiers can be more appropriate for different regions. Many efforts have been spent in looking for the best classifier and in fine tuning of the parameters. This requires knowledge of the domain and good expertise in fine tuning of the various classifiers. We propose a technique of voting with neural networks that does not require knowledge of the domain and avoids for a specific research of hyper-parameters values, because neural networks topology and the hyper-parameters are generated at random from a uniform distribution. The output of each neural network can be considered as a ranking over the probabilities that each sample belongs to a class, so the output of each neural network is a preference over the classes. We aggregate these preferences via different voting rules and we provide an empirical evaluation of the approach.