Estimating the joint probability density function of a dataset is a central task in many machine learning applications. In this work we address the fundamental problem of kernel bandwidth estimation for variable kernel density estimation in high-dimensional feature spaces. We derive a variable kernel bandwidth estimator by minimizing the leave-one-out entropy objective function and show that this estimator is capable of performing estimation in high-dimensional feature spaces with great success. We compare the performance of this estimator to state-of-the art maximum-likelihood estimators on a number of representative high-dimensional machine learning tasks and show that the newly introduced minimum leave-one-out entropy estimator performs optimally on a number of high-dimensional datasets considered.
Reference:
Van der Walt, C.M. & Barnard, E. 2017. Variable kernel density estimation in high-dimensional feature spaces. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), 4-9 February 2017, San Francisco, California, USA
Van der Walt, C. M., & Barnard, E. (2017). Variable kernel density estimation in high-dimensional feature spaces. Association for the Advancement of Artificial. http://hdl.handle.net/10204/9562
Van der Walt, Christiaan M, and E Barnard. "Variable kernel density estimation in high-dimensional feature spaces." (2017): http://hdl.handle.net/10204/9562
Van der Walt CM, Barnard E, Variable kernel density estimation in high-dimensional feature spaces; Association for the Advancement of Artificial; 2017. http://hdl.handle.net/10204/9562 .