Konstantin Shkurko

Konstantin Shkurko

Ph.D Student
School of Computing
University of Utah

Advisor: Erik Brunvand

School of Computing
50 S Central Campus Dr, RM 3190
Salt Lake City, UT 84112

kshkurko AT cs DOT utah DOT edu
kis9 AT cornell DOT edu

Semantic Image Retrieval Picture

A Radial Basis Function and Semantic Learning Space Based Composite Learning Approach to Image Retrieval

Konstantin Shkurko, Xiaojun Qi

ICASSP (International Conference on Acoustics, Speech, and Signal Processing), 2007

Abstract: This paper introduces a composite learning approach for image retrieval with relevance feedback. The proposed system combines the radial basis function (RBF) based lowlevel learning and the semantic learning space (SLS) based high-level learning to retrieve the desired images with fewer than 3 feedback steps. User's relevance feedback is utilized for updating both low-level and high-level features of the query image. Specifically, the RBF-based learning captures the non-linear relationship between the low-level features and the semantic meaning of an image. The SLS-based learning stores semantic features of each database image using randomly chosen semantic basis images. The similarity score is computed as the weighted combination of normalized similarity scores yielded from both RBF and SLS learning. Extensive experiments evaluate the performance of the proposed approach and demonstrate our system achieves higher retrieval accuracy than peer systems.

Files:     paper (pdf, 167 KB)     BibTex (bib, 1.47 KB)     presentation (pdf, 625 KB)

Acknowledgements: This work was supported by the National Science Foundation REU (Research Experience for Undergraduates).

Description: I have participated in the REU program in Computer Vision and Image Processing at the Department of Computer Science, Utah State University. I worked on Content-Based Image Retrieval (CBIR) using relevance feedback and have combined two very different approaches: radial basis function (RBF) network and semantic space. My mentor, Xiaojun Qi, and I have proposed a novel way of building the semantic space.

The basic idea was to search through an image database (in our case, 6000 COREL) categorized into different classes, like "Africa," "Elephants," "Busses," and return images similar semantically. RBF was used to home in on relevant images, by taking into account images marked by the user as relevant and non-relevant. The used features included: HSV color histogram, first three RGB color moments, and MPEG-7 edge histogram. The Semantic Space held semantic similarity between images, and was used to update semantic feature vectors. Combining both of the methods allowed to improve the results from averaging about 70 - 80% to high 90%, with 30 images returned.