The History Of The Recommendation System

Published Date: 02 Nov 2017

Recommendation System

ABSTRACT

In Today`s world information is generated at exponential rate, which poses a big challenge for the e-market industry i.e. how to offer best recommendation of an item to the user? To overcome these challenges, researchers have been putting a lot of efforts to evolve recommendation platform which can predict the best items that can be recommended to the end user. Mostly recommendation system implements knowledge discovery technique to provide accurate recommendation [1]. These recommendation systems generates the list of recommendation on the basis of two types of filtering process i.e. Collaborative filtering ,content based filtering and hybrid based filtering [2] .Researchers so far, has been successful to evolve such platform but there exist some open ended issues which needs to be addressed. In this article we will review the recommendation systems; the key advancement in recent times and the open challenges of the recommendation system. Based on our analysis of the state of the field, we identify some of the open research problems, and outline key challenges slowing the advance of the state of the art, and in some cases limiting the relevance of research to real-world applications.

Author Keywords

Recommendation system; collaborative filtering; content-based filtering; Hybrid filtering; push attack; nuke attack.

INTRODUCTION

An extensive class of web applications, which involve prediction of user responses to probable options, such a facility is called a recommendation system [3]. Obtaining recommendations from trusted sources is a critical component of the natural process of human decision making. In a recommendation-system application there are two classes of entities, which we shall refer to as users and

items. Recommendation systems use a number of different technologies. We can classify these systems into Content-based systems, Collaborative filtering systems and hybrid approaches [2]. We will discuss about these technologies in detail. Suggestions for books on Amazon, or movies on Netflix, are real world examples of the operation of industry-strength recommender systems. As Recommender Systems are being increasingly adopted by commercial websites, they have started to play a significant role in affecting the profitability of sellers. This has led to many unscrupulous vendors engaging in different forms of fraud to game recommender systems for their benefit. Fraud attacks like push attack, nuke attacks etc. proved to be a challenge for recommendation systems. Other challenges include cold-start and sparsity problems.

Collaborative Filtering methods can be further sub-divided into neighborhood-based and model-based approaches. Neighborhood-based methods are also commonly referred to as memory based approaches [2]. For example, a collaborative filtering recommendation system for television tastes could make predictions about which television show a user should like given a partial list of that user's tastes (likes or dislikes). Note that these predictions are specific to the user, but use information gleaned from many users. Content-based recommenders refer to approaches that provide recommendations by comparing representations of content describing an item to representations of content that interests the user. These approaches are sometimes also referred to as content-based filtering. For instance, given movie genre information, and knowing that a user liked "Star Wars" and "Blade Runner", one may infer a predilection for Science Fiction and could hence recommend "Twelve Monkeys". One simple approach is to allow both content-based and collaborative filtering methods to produce separate ranked lists of recommendations, and then merge their results to produce a final list. In order to do this we built a hybrid content-based collaborative filtering recommendation system [2].

Related Work

In this section we briefly present some of the research literature related to the recommendation system, open challenges related to recommendation system, attacks on the recommender system and address different technologies used by the recommender systems. Recommendation systems provide a filtered list of items to the end user on web applications such as Amazon, Netflix etc. Recommendation systems are classified in to two broad groups. Content-based systems, which examine properties of items recommended and Collaborative filtering systems, which recommend items based on similarity measures between item and users [3]. Prem Melville and Vikas Sindhwani explain Hybrid approach which combines both methods Collaborative and content based approaches [2]. Due to its implementation on large scale [3] researchers and even industry observed some open challenges related to recommendation system. According to Richard Mac Manus [5] some of the major problems in the recommender system are Lack of data, Changing data, changing user Preferences and Unpredictable items. Every system which is based on users input has to face security threats. Recommendation system also suffers from the security threats [10] such as Random attack, favorite item attack, and segmented attack. One of the major problems in online recommender system is shilling attacks. The shelling attacks could be noxious to collaborative filtering systems and can be detected based on the user rating patterns [6]. In order to reduce the threat of shilling to the recommender systems Shyong K. Lan & John Riedl performed twenty-four experiments and their results suggests the operators of the recommender system to concentrate on some of the rich areas like, Prefer Item-Item, Use Recommendation Metrics, Watch Metrics but worry anyway and protect new items.[7].

Background and Motivation

An exciting characteristic of recommender systems is that they draw interest of industry and businesses while it also draws a lot of attention from researchers. In spite of significant progress in the research community, and industry efforts to bring the benefits of new techniques to end-users, there still exists many issues that make personalization and adaptation a complex procedure for the user.

Figure 2 [11]

Research activities still often focus on problems, such as accuracy improvements of current techniques, sometimes with ideal hypotheses, or tend to overspecialize on a few applicative problems. Thus, we may have reached a good point to take a step back to seek perspective in the research done in recommender systems. We propose an analytic outlook on new research directions, or ones that still require substantial research, with a special focus on the some open ended issues which needs to be addressed. Due to its vast area implementation and interesting open problem, recommendation systems encourage us to go in-depth and gain more knowledge about this topic.

Open Problems

Let us discuss some of the major open problems of recommendation system in-brief

 Lack of Data [5]: The biggest issue is the system needs a lot a data in order to give effective recommendations. The more data the recommender system has to work with, the better the chances of getting good recommendations. But lots of users are required in order to get a lot of data for recommendations

 Cold-start Problem [5]: New items and new users give recommender system a significant challenge. This problem comes where an item cannot be recommended unless it is rated by any of the user previously. This problem is not limited to new items alone but also to the items which is partially detrimental to users with different tastes.  The new item problem is often referred to the first rater problem.

 Changing User Preferences [5]: The issue here is that user often changes his preferences while browsing. Let's take an example in explaining the issue. If a user is browsing Amazon in search of new books, the next day he wants to buy some gift. So, the preferences of the user differ frequently.  The Recommender system fails in handling such cases.

Unpredictable items [5]: About $1 million prize  was offered by Netflix to a third party to deliver a collaborative filtering algorithm which can improve  Netflix’s recommendations algorithm by 10%.The issue here is with eccentric movie. People may either like or dislike this type of movies. One example of such movies is Napoleon Dynamite. This type of movie fails in the recommender system, because the user reaction tends to be unpredictable.

Security problem [10]: As recommender systems are increasingly being adapted by commercial websites, the recommender systems are playing a major role in product selling which in turn gets good profit’s to the company. This indeed tends to unscrupulous vendors who are involving in different forms of fraud to game recommender system for their benefit. Generally, the attacks are classified into two categories, Push attacks or Nuke attacks. The attempt to inflate the perceived  desirability of their own products are considered to be push attacks and the attempt to lower the ratings of the competitors’ products are considered to be nuke attacks. These attacks are broadly studied as shilling attacks or profile injection attacks respectively. Some of the attacks are mentioned below:

Figure 3 [9]

Random attack [10]:  An attack model in which profiles consist of random values (except of course for a positive rating given to the pushed item). Specifically, ratings that are assigned to the corresponding items are random values within the rating scale with a distribution centered around the mean for all user ratings across all items. The knowledge required to mount such an attack is quite minimal, especially since the overall rating mean in many systems can be determined by an outsider empirically.

Favorite Item Attack [10]: The favorite items are the ones whose ratings are greater than the user’s average rating to all items. These items are assigned maximum rating value together with the target item. The other items in the database are assigned ratings at random or based on other criteria. It will affect significantly when the non-favored items are assigned the lowest possible rating. This attack is not particularly practical from a knowledge cost standpoint, but provides an upper bound on the effectiveness of other attacks focused on user characteristics.    

Segmented Attack [10]: This attack requires very limited knowledge about the system and the users. An attacker needs to know only a group of items well-liked by the target segment and needs to build profiles containing only those items. For example some movies that are common to a segment of users can be assigned the maximum rating value together with the target item. Similar to Favorite Item attack, it will affect significantly when the other items are assigned the lowest possible rating.

Apart from above three attacks, there are other attacks that were identified in the past.

Future Work

As a part of future work, we will do a research on one of the recommendation problems and figure out ways to deal with the same. As of now, all the mentioned problems are open problems and there are no best solutions available in the literature as per our knowledge. We will be working to deal with one of the serious problems, so as to give a value add to the field of "Intelligent Interactive Systems". This might help researchers in this domain to deal with other related problems in Recommender System.