The match score image: A visualization tool for image query refinement

Abstract

We present a visualization technique designed to facilitate iterative refinement of content-based image queries, particularly example-based specification. The technique operates on scores produced by region-based matching algorithms, including texture matching and template matching. By mapping match scores to color, then compositing with the original image, we provide the user with the "goodness" of match for each region and simultaneously with the original image information. There are several ways in which the match score image can be used to enhance the query refinement process including: facilitating the selection of both positive and negative examples, guiding the selection of thresholds, and enabling exploration of the effect of other parameter values on match algorithm performance. The usability of this visualization technique is highly dependent on choice of score-to-color mapping parameters including continuous vs. discrete, hue range, saturation, lightness, and transparency. We provide some heuristics for selecting these values. Although usable for photographic images, the match score image is particularly useful in application domains such as remote sensing and medical imaging, where particular subregions of large images are sought, rather than entire images.