Communication and Information Sciences (CIS) PhD program. LIS 678 / CIS 702 Communication/Information Technologies. Professor Luz Marina Quiroga.
Image Retrieval
1
Image Retrieval – State of the Art and future challenges Philipp Jordan University of Hawai'i at Manoa Communication and Information Sciences (CIS) PhD program LIS 678 / CIS 702 Communication/Information Technologies Professor Luz Marina Quiroga 14 December, 2011
Image Retrieval
2 Abstract
One important area of Multimedia Information Retrieval is Image Retrieval. Outlining the necessity for effective and efficient Image Retrieval methods, this paper benchmarks and evaluates qualitative three exemplary systems, namely Google Image search, ALIPR and IBM´s QBIC. The conclusions include a summary of the benefits and challenges of content-based Image Retrieval and present the future work of the field. Keywords: Multimedia Information Retrieval, Image Retrieval.
Image Retrieval
3
Image Retrieval – State of the Art and future challenges Introduction Image Retrieval is a subfield of Multimedia Information Retrieval which evolved significantly in the last 10 years. This paper will outline the basic approaches of Image Retrieval and the theoretical foundation underneath. Furthermore, a qualitative assessment of three deployed systems reveals ambivalent results. The final conclusions include suggestions for the future work of the field. Image Retrieval The digital age comes along with the need to organize and retrieve information in new ways. Especially Image Retrieval, both within a personal or professional context, has been lately paid much attention too. Digital photo collections, digitalization of paintings or effective mobilebased Image Retrieval are the preferred areas of application nowadays (Lew et al 2006). Within the literature study, two main concepts for Image Retrieval have been identified. Content-based Image Retrieval, aka query-by-image-content or content-based information retrieval is any technology that aims to organize picture archives by their visual content. On the other side, concept-based image indexing refers to retrieval from text-based indexing of images that may employ keywords, subject headings, captions, or natural language text (Schmitt 2006). An effective content-based image search and retrieval is usually employing certain algorithms, calculating some degree similarity or distance of the color, object or shape of the queried image while the general fundamental problem of content-based image retrieval is the bridging of the sematic gap between humans and computers (Datta et al. 2009).
Image Retrieval
4
Qualitative Retrieval In the following section, three available systems for content-based Image Retrieval have been randomly selected by the author for a qualitative evaluation. QBIC IBM IBM offers on the State Hermitage Museum a query by content-related search interface, in particular a color and layout search. Using the layout search creating a generic scenery with a blue sky, a yellow sun and a plain green field, QBIC revealed to successfully retrieving similar images, such as paintings from cities or natural viewpoints. Hence, the results also contained false positives, such as paintings of persons when querying for a landscape. Overall, the performance of QBIC is strongly scattered. Google Image search Using 4 self-taken pictures of a pineapple, a sunset at the beach and two distinct buildings, Google Image search has been tested and unveil itself as very effective. Two buildings have been correctly identified by their name (Ashton Waikiki Circle Hotel and the Dole Planation), while for two photographs of a pineapple plant and a sunset revealed similar (in terms of color and shape), but not identical pictures. ALIPR ALIPR is an automatic image annotation engine which allows the user to upload a picture to the website and then recommends tags to describe the same. Using the same 4 photos as in the Google Image search, the system ALIPR predicted correct tags (such as building for the Ashton Waikiki Circle Hotel) but also false-positives. Ultimately, the user is in responsibility to select the correct tags from the recommended choices.
Image Retrieval
5
Conclusions This paper tried to unveil the current development stage of Image Retrieval as a subcategory of Multimedia Information Retrieval. Content-based Image Retrieval seems a desirable field as the verbalization of the query is no longer required. The current challenges of the field are the proper handling of private and security issues, such as personalization and face identification. Furthermore, a certain impedition of the scientific progress due to the reluctance of companies publishing their retrieval algorithms is another challenge. The future work can be outlined within 5 distinct areas, in particular sematic searches with emphasis on the concept detection of images with complex backgrounds, multi-modal analysis and retrieval algorithms exploiting synergy effects of the content and context of the image, experiential multimedia exploration systems, interactive search and relevance feedback systems and finally, evaluation with emphasis on representative test sets and usage patterns. References Chen, H.-L. (1999) E.M. Rasmussen. H.-L. Chen and E.M. Rasmussen , Intellectual access to Images , Library Trends 48(2) (1999) 291-302 .
Datta, R., Jia Li, & Wang, J. (2009). Exploiting the Human-Machine Gap in Image Recognition for Designing CAPTCHAs. IEEE Transactions on Information Forensics and Security, 4(3), 504–518. doi:10.1109/TIFS.2009.2022709
Lew, M. S., Sebe, N., Djeraba, C., & Jain, R. (2006). Content-based multimedia information retrieval. ACM Transactions on Multimedia Computing, Communications, and Applications, 2(1), 1–19. doi:10.1145/1126004.1126005
Image Retrieval Schmitt, I. (2006). Multimedia Information Retrieval. Brandenburgische Technische Universität Cottbus - Institut für Informatik, https://docs.google.com/viewer?url=http://inka.htwberlin.de/mp06/docs/Schmitt.pdf&pli=1.
6
