Review On Image Retrieval Systems Computer Science Essay

Published Date: 02 Nov 2017

Ms. Jyoti Dattatraya Gavade

VPCOE, Baramati,Maharashtra , India

[email protected]

Mrs. Gyankamal J. Chhajed

VPCOE, Baramati,Maharashtra , India

[email protected]

Abstract- In this paper we have reviewed and analyzed different image retrieval systems. The purpose of this survey however, is to provide an overview of the functionality of temporary image retrieval systems in terms of technical aspects: querying, relevance feedback, features, matching measures, indexing data structures, and result presentation.We have reviewed different techniques like text based retrieval ,content based retrieval, image annotation to get images captured by digital camera. The classification techniques such as k-NN,SVM,Decision stump,Manifold Ranking,Hash Encoding Algorithm followed by a suitable relevant feedback model via cross domain learning , GMI-SVM , Laplacian Regularized Least Squares(LapRLS), Search Result Clustering(SRC)Algorithm , Biased Discriminative Eclidean embedding (BDEE)to refine the image retrieval result of consumer photos. After thorough study , this review also claims that most systems uses low level features and only few uses high level semantically meaningful features and the image retrieval results affect due to this semantic gap . The semantic gap is often regarded as a major problem in the field of image retrieval research. The comparative chart presents the details of different image retrieval system and addresses the features to be considered for designing any image retrieval utility.

Keywords- Text based retrieval, Content based retrieval, Image annotation, Classifiers, Relevant feedback(Query refinement strategies)

I. INTRODUCTION

There is lots of increasing interest in the world of digital photography. Now a day's every person has a digital camera or mobile. Every person has rights to capture photos of beautiful nature and surrounding. These captured images are not consisting of semantic concepts such as proper name like web images. Person may be put a proper name to the captured image or may not be and these images are organized in folders without providing indexing. So we remove the difficulty of retrieving images from personal collections different image retrieval techniques are present.

How to retrieve images?

And also with the ever-growing number of images on the Internet, retrieving relevant images from a large collection of database images has become an important research topic.

It is well known that the major problem in image retrieval is the semantic gap between the low-level features (color, texture, shape, etc.) and the high-level semantic concepts. So it is very important question , How to minimize "semantic gap" as shown in figure 1?

Fig 1: Semantic gap problem

All of Proposed system consist web image database and classifiers to satisfy users demand for proper relevant image retrieval. To refine the result along with popular Wordnet onthology from which we get relevant and irrelevant images and these set is use to build classifier. These Classifiers such as k-nearest Neighbor which compute average distance, Support vector machine which considers multiple feature dimensions gives initial retrieval results. To refine the retrieval result of personal photos consisting of feature distribution may differ in web images and personal photos, relevant feedback approach is proposed.

II. BACIS IDEA OF IMAGE RETRIEVAL SYSTEM

Let us consider the basic idea of image retrieval.

General goal of image retrieval system are:

- That is able to process natural language query.

- That is able to search among annoted and non-annoted images.

- That takes into account human visual perception.

- That processes various features (color, texture, shapes) .

Generally most of current image retrieval systems use low-level features such as color, texture and shape because they are extracted by a machine automatically as shown in Table 1.

Table 1: Overview of commonly used features in IR[7]

Colour

Histogram,Color co-occurrence histograms

Shape

Segmentation & Contour extraction followed by: counter matching, moments, template matching

Texture

Directionality,periodicity,randomness,Fourier domain characteristics, random fields

Other

Wavelet coefficient,eigenimages,edge-map of user made sketch, image context vectors

III. IMAGE RETRIEVAL TECHNIQUES

The image retrieval system consists of three approaches

a) Text based image retrieval

b) Image annotation before TBIR

c) Content based image retrieval

d) Hybrid of text and image based image retrieval

a) Text based image retrieval :

1] Textual query-based consumer photo retrieval system:

In this paper Yiming Liu, Dong Xu, Member, IEEE, Ivor Wai-Hung Tsang, and Jiebo Luo, proposes a novel methodology for Textual query-based consumer photo retrieval system as shown in figure 3.

The process can be done as:

a) They introduce how to retrieve consumer photo considering millions of web images with their rich textual descriptions.

b) They perform integration of large database and Wordnet to get relevant and irrelevant images based on textual query. After that apply classification techniques such as kNN, SVM, Decision stumps.

c) To refine the retrieval result of personal photos consisting of feature distribution may differ in web images and personal photos, we propose cross-domain approach.

Advantages:

1) Images can be retrieved without using image annotation process.

2) Framework is efficiently used for large scale consumer photo retrieval.

2] Bag-based ranking:

In this paper, Lixin Duan,Wen Li,Ivor Wai-Hang Tsang and Dong Xu proposes a novel methodology for improving web image search by bag based reranking[2].The proposed methodology used text query to get relevant images and then performed reranking using visual features.

The process is done as follows:

a) Combining of both visual and textual features, they form cluster of relevant images. Each cluster can be considering as a "bag" and the images present in bag are treat as "instances," then apply multi-instance (MI) learning problem. b) They use mi-SVM as MI learning method so that can be readily incorporated into bag-based reranking framework. Observing that at least a certain portion of a positive bag is of positive instances while a negative bag might also contain positive instances,

c) Further use a more suitable generalized MI (GMI) setting for this application. To address the ambiguities on the instance labels in the positive and negative bags under this GMI setting, they develop a new method referred to as GMI-SVM to enhance retrieval performance by propagating the labels from the bag level to the instance level. To acquire bag annotations for (G)MI learning, they propose a bag ranking method to rank all the bags according to the defined bag ranking score. The top ranked bags are used as pseudo positive training bags, while pseudo negative training bags can be obtained by randomly sampling a few irrelevant images that are not associated with the textual query.

Advantages:

1) The framework with automatic bag annotation can achieve the best performances compared with existing image reranking methods.

2) GMI-SVM can achieve better performances when using the manually labelled training bags obtained from relevance feedback.

Disadvantages

1) Labels of relevant training images are quite noisy so the constraints on positive bags may not always be satisfied in this application.

3] Manifold- Ranking Algorithm:

In this paper, J. He, M. Li, H. Zhang, H. Tong, and C. Zhang proposes a novel methodology Manifold-

Ranking Based Image Retrieval [3]. The proposed methodology is based on the following steps:

1) They propose a novel transductive learning framework for image retrieval based on a manifold ranking algorithm- here first weighted graph is formed using kNN approach and assign a positive ranking score to each query and zero to remaining points.

2. Then design and investigate different schemes for utilizing the positive and combination of positive ,negative relevance feedback to improve the retrieval result

3. Finally use active learning methods to speed up the convergence to the query concepts.

Advantages:

1) Processing time can be greatly reduced.

2) It reduced scale of weighted graph to form a small graph.

Disadvantages:

1) It degrades the performance of Relevance feedback.

b) Image Annotation

Image annotation as the application of computer vision for retrieving images is also known as automatic image tagging or linguistic indexing. Using tagging or annotation method computer system directly assigns captioning or keywords to a digital image[5].

It is commonly used to classify images according to the high-level semantic concepts. It is generally used as an intermediate stage for TBIR image retrieval and must be performed before it because the semantic concepts are analogous to the textual terms that describe document contents[9].When a input text given by user are not present in the current set of vocabularies then user needs to perform another annotation to consumer photos.

It is classified into Learning-based methods and Web data-based methods.

a)In learning-based methods first the classifiers worked on a large labelled training data and finally used to detect the presence of the concepts in any test data[7].

Disadvantage : Time consuming and Expensive human annotation.

b) Web data-based methods: Annotations on general images can be perform by Web data-based methods[3].For raking of images it does not provideany metric.

Disadvantage :

For raking of images it does not provide any metric.

1)Tag Ranking

It is proposed to rank the existing tags.It is performed by considering relevance scores of contents present in Flickr images.

The relevancy of the tags can be learned by counting the tag votes of the photos which are visually similar.[8]

Disadvantage:

KDE algorithm is first used to obtain the initial estimation of tag relevance,finally for tag refinement a random walk algorithm is used.

But they did not utilize negative training data and can't create new tags.

2)Object recognition using k-NN

If the database is big enough then object must be find,with high probability,images visually close similar to a query image.They also consider similar objects with similar scenes and ,must be arranged arranged in similar spatial configurations.[3]

The problem addressed in this paper is that even if dataset is big,k-NN efficiently perform recognition of object.

Advantages :

1) It is debug and implement easily because the process is transparent.

2)If neighbours analysis is good then k-NN is very effective.

3)It improves the accuracy by using noise reduction techniques.

Disadvantages :

1)If training dataset is so large then run time performance of k-NN is poor.

2) k-NN is very sensitive to irrelevant or redundant features.

3) Tag based Image Retrieval:

In this paper Lin Chen, Dong Xu, Ivor W. Tsang, Jiebo Luo Tag-based Image Retrieval Improved by Augmented Features and Group-based Refinement [4] In this paper, they propose a new tag-based image retrieval framework to improve the retrieval performance of a group of related personal images captured by the same user within a short period of an event by leveraging millions of training web images and their associated rich textual descriptions.

a) For any given query tag the inverted file method is employed to automatically determine the relevant and irrelevant training web images that are associated with the query tag.

b) Using these relevant and irrelevant web images as positive and negative training data respectively, they propose a new classification method called SVM with Augmented Features (AFSVM) to learn an adapted classifier by leveraging the pre-learned SVM classifiers of popular tags that are associated with a large number of relevant training web images.

c) For refinement process, they propose to use the Laplacian Regularized Least Squares (LapRLS) method to further refine the relevance scores of test photos by utilizing the visual similarity of the images within the group.

.

Advantages:

1) The technique capture the geometry of the data points in the high-dimensional space.

4] Auto-annotation:

In this paper Xin-Jing Wang, Lei Zhang, Feng Jing, Wei-Ying Ma presents AnnoSearch, a novel way to annotate images using search and data mining technologies.

a) Firstly, at least one accurate keyword is required to enable text-based search for a set of semantically similar images.

b) Then content-based search is performed on this set to retrieve visually similar images. At last, annotations

are mined from the descriptions (titles, URLs and surrounding texts) of these images.

c) It worth highlighting that to ensure the efficiency, high dimensional visual features are mapped to hash codes

which significantly speed up the content-based search process.

Advantages:

1) Searching for semantically and visually similar images on the Web and mining annotations from them

2) Annotating with unlimited vocabulary, which is impossible for all existing approaches.

Disadvantages:

Do not provide metric to rank the images

c)Content based image retrieval:

Now a days very popular image retrieval technique is the Content-based image retrieval (CBIR) which used visual information. In this they must have to give query as an example instead of text query. It is also known as query by image content (QBIC).[12].The retrieval process consist of the contents of the image such as textures, shapes,colors and other information of image itself.[3].

Relevence feedback can be proposed in CBIR systems to recover the semantic gap.In RF search results will be improved or refine the results based on whether the results are related not releted or neutral to search query then repeating the search with the new information. .

SVM-based relevance feedback methods were proposed[3][6].

Advantages:

4)Search images without any information

E.g. possibal to find images showing a particular person, given

a suitable face detection

5)Find similar Images

E.g:find images with certain color , find duplicates to clean up the set of images.

Disadvantages:

1)The gap between the low-level visual features and the high-level semantic concepts.

1] Biased Discriminative Euclidean Embedding (BDEE)

In this paper Wei Bian and Dacheng Tao presents Biased Discriminate Euclidean Embedding for

Content-Based Image Retrieval has represented images by low-level visual features. They have designed a mapping to select the effective subspace from for separating positive samples from negative samples based on a number of observations. They have proposed the Biased Discriminative Euclidean Embedding (BDEE) which parameterizes samples in the original high-dimensional ambient space to discover the intrinsic coordinate of image low-level visual features.

Advantages:

1) It preserves both the intraclass geometry and interclass discrimination

2) It is superior to the popular relevance feedback dimensionality reduction algorithms.

3) Its extension considers the unlabelled samples.

2)

Support Vector Machine Active Learning for Image Retrieval :

In this paper S. Tong and E. Chang presents Support Vector Machine Active Learning for Image Retrieval Mostly CBIR systems returns semantically relevant images to the user's query image so

the number of techniques present in CBIR vary depending on the application, but result images should all share common elements with the provided example.. However as persons point of view , it is more convenient and natural for a user to retrieve images using a query as text.

The early relevance feedback method directly adjust weights of various features . SVM-based relevance feedback methods were proposed[3][6].

Disadvantages:

2)It degrade the retrieval performance of the techniques considering limited number of feedback images.

IV. ANALYSIS

Sr.No

Input

Index Implementation

Classifiers

Query Refinement strategies

Database used

Features Used

EVALUATION PARAMETER

PERFORMANCE

RESULTS

[1]

Text

Inverted Index Using Wordnet

kNN,SVM,

Decision Stumps

Relevance Feedback with CDRR

Training Data-1.3million photos forum Photosig,

Test data-Kodak Photo

NUS-WIDE

precision

precision 4.7 %- Kodak dataset and 13.5 %

NUS-WIDE datasets

[2]

Text

Inverted File Method

k-means Clustering method

GMI-SVM

NUs_WIDE dataset Flickr Images

[3]

Text

Manifold Ranking Algorithm

KNN

Relevance feedback with active learning methods

Corel Image Gallery

[4]

[5]

[6]

Text

Inverted File Method

AFSVM

Laplacian Regularized Least Squares(LapRLS)

Training Data-1.3million photos forum Photosig,

Test data-Kodak Photo, Flickr Photo

[7]

Text

+

Image

Hash Map

Hash Encoding Algorithm

Search Result Clustering(SRC)Algorithm

2.4 million photos forum Photosig

Precision

And

Recall

Precision=38.14%

Recall=22.95

[8]

Image

Euclidean measure

Biased Discriminative

Eclidean embedding(BDEE)

Biased Discriminative

Eclidean embedding(BDEE)

Coral

Image Gallery

color, we selected hue, saturation, and value histogram

Precision.

And standard deviation

Average Precision

= 0.32

for 9 RF iterations

9

v.CONLCUSION

In this paper we have reviewed and analyzed different methods to retrieve images capture by digital camera or mobile which do not have high level semantics concept. We have reviewed different techniques like text based retrieval based on wordnet,,SVM,kNN, auto-annotation by hashmap,clustering , Augmented Features and Group-based Refinement approach, Manifold- Ranking Algorithm ,bag-based reranking method for efficient retrieve images, Biased Discriminate Euclidean Embedding for Content-Based Image Retrieval.

VI.REFERENCES

[1] Textual Query of Personal Photos Facilitated by Large-Scale Web Data Yiming Liu, Dong Xu, Member, IEEE, Ivor Wai-Hung Tsang, and Jiebo Luo, Fellow, IEEE

[2] Improving Web Image Search by Bag-Based Reranking Lixin Duan, Wen Li, Ivor Wai-Hung Tsang, and Dong Xu, Member, IEEE

[3] J. He, M. Li, H. Zhang, H. Tong, and C. Zhang, "Manifold-Ranking Based Image Retrieval," Proc. ACM Conf. Multimedia, 2004

[4] Lin Chen, Dong Xu, Ivor W. Tsang, Jiebo Luo Tag-based Image Retrieval Improved by Augmented Features and Group-based Refinement

[5] AnnoSearch: Image Auto-Annotation by Search Xin-Jing Wang, Lei Zhang, Feng Jing, Wei-Ying Ma

[6] Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval Wei Bian and Dacheng Tao, Member, IEEE

[7] New Methods for Image Retrieval Zoran Peˇcenovi´c, Minh Do, Serge Ayer, Martin Vetterli Laboratory for Audio-Visual Communications, Swiss Federal Institute of Technology (EPFL) CH-1015 Lausanne, Switzerland

[8] Y. Rui, T.S. Huang, and S. Mehrotra, "Content-

Based Image Retrieval with Relevance Feedback in

Mars,"Proc. IEEE Int Conf. Image Processing, 1997.

9) S. Tong and E. Chang, "Support Vector Machine Active

Learning for Image Retrieval," Proc. ACM Conf.

Multimedia, 2001.

10) L. Chen, D. Xu, I.W. Tsang, and J. Luo, "Tag-Based

Web Photo Retrieval Improved by Batch Mode Re-

Tagging," Proc. IEEE Conf. Computer Vision and

Pattern Recognition, 2010.