Leveraging Gaussian process approximations for rapid image overlay production

Abstract

Machine learning models trained using images can be used to generate image overlays by investigating which image areas contribute the most towards model outputs. A common approach used to accomplish this relies on blanking image regions using a sliding window and evaluating the change in model output. Unfortunately,this can be computationally expensive,as it requires numerous model evaluations. This paper shows that a Gaussian process approximation to this blanking approach produces outputs of similar quality,despite requiring significantly fewer model evaluations. This process is illustrated using a user-driven saliency generation problem. Here,pairwise image interest comparisons are used to infer underlying image interest and a Gaussian process model trained to predict the interest value of an image using image features extracted by a convolutional neural network. Interest overlays are generated by evaluating model change at blanking image regions selected using the prediction uncertainty of a Gaussian process regressor.