An Ensemble Feature Selection Framework Integrating Stability

Abstract

Ensemble feature selection has drawn more and more attention in recent years. There are mainly two strategies for ensemble feature selection, namely data perturbation and function perturbation. Data perturbation performs feature selection on data subsets sampled from the original dataset and then selects the features consistently ranked highly across those data subsets. Function perturbation frees the user from having to decide on the most appropriate selector for any given situation and works by aggregating multiple selectors. Our study showed that function perturbation resulted in a low stability. We therefore propose a framework, Ensemble Feature Selection Integrating Stability (EFSIS), combining these two strategies and integrating stability during the aggregation of selectors. Empirical results indicate that EFSIS highly improves stability and meanwhile, maintains the prediction accuracy.