Implement And Evaluation Of A Recommender System

Published Date: 02 Nov 2017

In the recent years, the Web has undergone a tremendous growth regarding both content and users. This has lead to an information overload problem in which people are finding it increasingly difficult to locate the right information at the right time.

Recommender systems have been developed to address this problem, by guiding users through the big ocean of information. Until now, recommender systems have been extensively used within music domain and communities where items like movies, music and articles are recom­mended. More recently, recommender systems have been deployed in online music players, recommending music that the users probably will like.

This thesis will present the design, implementation, testing and evaluation of a recommender system within the music domain, where three different approaches for producing recommendations are utilized.

Testing each approach will be done by first conducting live user experiments and then measure recommender precision using online analysis. Our results show that the functionality of the recommender system is satisfactory, and that recommender precision differs for the three filtering approaches.

Introduction

The World Wide Web contains an enormous amount of information. In January 2005 the number of pages in the publicly index able (Selberg, 1999) web was exceeding 11.5 billion (Gulli & Signorini, 2005). Recent statistics also show that the number of Internet users is high and rapidly growing. Statistics from September 18th 2006 shows that 17% of the world's population uses the Internet and that the number of users has grown with over 200% from 2000 to 2006 (Angil, 2008). The tremendous growth of both information and usage has lead to a so-called information overload problem in which users are finding it increasingly difficult to locate the right information at the right time (Resnick, et al., 1994). As a response to this problem, much research has been done with the goal of providing users with more proactive and personalized information services.

Recommender systems have proved to help achieving this goal by using the opinions of a community of users to help individuals in the community more effectively identify content of interest from a potentially overwhelming set of choices (Resnick & Varian, 1997). Two recommendation strategies that have come to dominate are content-based and collaborative filtering. Content-based filtering relies on rich content descriptions of the items that are being recommended (Melville, et al., 2001), while collaborative filtering recommendations are motivated by the observation that we often look to our friends for recommendations (Melville, et al., 2001).

Systems using recommendations have been developed in various research projects. The system called Tapestry (Goldberg, et al., 1992) is often associated with the genesis of computer-based recommendation systems. Later, several research projects have focused on recommender systems, either by introducing new concepts, or by combining old concepts to make better systems.

Recommender systems have also been deployed within commercial domains, for example in E-Commerce applications. A well-known example is Amazon, where a recommender system is used to help people find items they would like to purchase. Many online com- munities within the movie domain use recommender systems to gather user opinions on movies, and then produce recommendations based on these opinions. Examples are MovieFinder2 and Movielens3. New popular music services like Pandora4 and Last.fm5 also make use of recommendations to configure personalized music players.

Background

Recommender Systems are software tools and techniques providing suggestions for items to be of use to a user (Mahmood & Ricci, 2009). The suggestions relate to various decision-making processes, such as what items to buy, what music to listen to, or what online news to read.

"Item" is the general term used to denote what the system recommends to user. Recommender system normally focuses on a specific type of item (e.g., CDs, or news) and accordingly its design, its graphical user interface, and the core recommendation technique used to generate the recommendations are all customized to provide useful and effective suggestions for that specific type of item.

Recommender systems are primarily directed towards individuals who lack sufficient personal experience or competence to evaluate the potentially overwhelming number of alternative items that a Website, may offer (Resnick & Varian, 1997). A case in point is a book recommender system that assists users to select a book to read. In the popular Web site, Amazon.com, the site employs a RS to personalize the online store for each customer (Jannach, 2006). Since recommendations are usually personalized, different users or user groups receive diverse suggestions. In addition there are also non-personalized recommendations. These are much simpler to generate and are normally featured in magazines or newspapers. Typical examples include the top ten selections of books, CDs etc. While they may be useful and effective in certain situations, these types of non-personalized recommendations are not typically addressed by RS research.

In their simplest form, personalized recommendations are offered as ranked lists of items. In performing this ranking, Recommender systems try to predict what the most suitable products or services are, based on the user’s preferences and constraints. In order to complete such a computational task, recommender systems collect from users their preferences, which are either explicitly expressed, e.g., as ratings for products, or are inferred by interpreting user actions. For instance, a recommender system may consider the navigation to a particular product page as an implicit sign of preference for the items shown on that page.

Recommender systems development initiated from a rather simple observation: individuals often rely on recommendations provided by others in making routine, daily decisions (Mahmood & Ricci, 2009).. For example it is common to rely on what one’s peers recommend when selecting a book to read; employers count on recommendation letters in their recruiting decisions; and when selecting a movie to watch, individuals tend to read and rely on the movie reviews that a film critic has written and which appear in the newspaper they read.

Aims of Recommender Systems

In the previous section we defined recommender systems as software tools and techniques providing users with suggestions for items a user may wish to utilize. Now we want to refine this definition illustrating a range of possible roles that a RS can play. First of all, we must distinguish between the roles played by the RS on behalf of the service provider from that of the user of the RS. For instance, a travel recommender system is typically introduced by a travel intermediary (e.g., Expedia.com) or a destination management organization (e.g., Visitfinland.com) to increase its turnover (Expedia), i.e., sell more hotel rooms, or to increase the number of tourists to the destination (Ricci, 2002). Whereas, the user’s primary motivations for accessing the two systems is to find a suitable hotel and interesting events/attractions when visiting a destination.

In fact, there are various reasons as to why service providers may want to exploit this technology:

• Increase the number of items sold.

This is probably the most important function for a commercial RS, i.e., to be able to sell an additional set of items compared to those usually sold without any kind of recommendation. This goal is achieved because the recommended items are likely to suit the user’s needs and wants. Presumably the user will recognize this after having tried several recommendations. Non-commercial applications have similar goals, even if there is no cost for the user that is associated with selecting an item. For instance, a content net- work aims at increasing the number of news items read on its site.

In general, we can say that from the service provider’s point of view, the primary goal for introducing a RS is to increase the conversion rate, i.e., the number of users that accept the recommendation and consume an item, compared to the number of simple visitors that just browse through the information.

• Sell more diverse items.

Another major function of a RS is to enable the user to select items that might be hard to find without a precise recommendation. For instance, in a movie RS such as Netflix, the service provider is interested in renting all the DVDs in the catalogue, not just the most popular ones. This could be difficult without a RS since the service provider cannot afford the risk of advertising movies that are not likely to suit a particular user’s taste. Therefore, a RS suggests or advertises unpopular movies to the right users

• Increase the user satisfaction.

A well designed RS can also improve the experience of the user with the site or the application. The user will find the recommendations interesting, relevant and, with a properly designed human-computer interaction, she will also enjoy using the system. The combination of effective, i.e., accurate, recommendations and a usable interface will increase the user’s subjective evaluation of the system. This in turn will increase system usage and the likelihood that the recommendations will be accepted.

• Increase user fidelity.

A user should be loyal to a Web site which, when visited, recognizes the old customer and treats him as a valuable visitor. This is a nor- mal feature of a RS since many Recommender systems compute recommendations, leveraging the information acquired from the user in previous interactions, e.g., her ratings of items. Consequently, the longer the user interacts with the site, the more refined her user model becomes, i.e., the system representation of the user’s preferences, and the more the recommender output can be effectively customized to match the user’s preferences.

• Better understand what the user wants.

Another important function of a RS, which can be leveraged to many other applications, is the description of the user’s preferences, either collected explicitly or predicted by the system. The service provider may then decide to re-use this knowledge for a number of other goals such as improving the management of the item’s stock or production. For instance, in the travel domain, destination management organizations can decide to advertise a specific region to new customer sectors or advertise a particular type of promotional message derived by analyzing the data collected by the RS (transactions of the users).

We mentioned above some important motivations as to why e-service providers introduce recommender systems but users also may want a recommender system, if it will effectively support their tasks or goals. Consequently a recommender system must balance the needs of these two players and offer a service that is valuable to both.

Problem Definition and Focusing Area

This thesis shall focus on development and evaluation of a recommender system within music domain. Different approaches for computing recommendations will be designed, implemented and tested with real users. Evaluation will be done by assessing the system functionality and comparing the recommender precision obtained by each approach.

Interpretation of research

Throughout the last years, recommender systems have been deployed in various personalized music players. One reason for the success behind these players is due to their ability to produce recommendations that accurately suits their users. By developing and testing different variants of a music player using standard recommendation strategies, we might be able to discover how the different techniques influence recommender precision.

We also conjecture that the standard strategies are not always sufficient to reflect a person’s preference, where preference often is context dependent (Daniel, 2005). One important aspect of a person’s context is mood. By integrating a mechanism for mood filtering into the music recommender system, it may be possible to give recommendations that better suits a person’s often varying music preference.

Three variants of the recommender system will be tested using content-based filtering, collaborative filtering and contextual collaborative filtering respectively. Testing includes user experiments, where the users evaluate and listen to recommended music while the system receives user feedback. Since listening to a vast variety of music generally takes time, we conjecture that the users normally will test the system during the week while studying or working. After testing the system, user feedback will be used to calculate recommender precision. Finally, the results will be presented and evaluated.

System overview

In this thesis, I will develop a centralized recommender system for music. An overview of the system is, clients communicate with a web server over the Internet. The web server provides a music service. After receiving song evaluations from the clients, the server will produce and provide the clients with personal music recommendations. Each recommendation consists of a play list with information about the music, and where the music is located.

Intellectual Challenge

7.1. Literature Review Intellectual Challenge

There will be several challenges in conducting this research. Namely the most obvious ones at this time would be in conducting the literature review of this area. As a matter of fact, literature from two scopes of studies ought to be covered and thoroughly assessed to gain a fruitful insight, one being the strands of studies done on Online Collaboration while also the other being studies done or continuing on methods, means and frameworks done on Web Development as a process. Not to mention, this said literature review ought to cover related areas while treeing in and out of all perspectives any study has to offer.

Moreover, the aspect of thinking critically based on the findings made both by the literature review and the future results of the proposed surveys suggested by this document would be another challenging task, since it would require a lot of out of the box thinking as well as intuitive writing on the same regard. This would be a difficult feat since various studies have already been conducted which would probably leave a very narrow space of approach on the dissertation.

.

This would also mean that results from certain locales would deem different patterns given into consideration of various factors that would come into play and this will have to be recognized before hand to deliver a more meaningful data set.

7.2. Time and Schedule Intellectual Challenge

The time provided is also another pressing concern, planning the project while keeping up with the schedule would be another "tough" task. This being said the emphasis on working with supervisors and working with them would be helpful as well as pose several other pressing matters of the study’s nature. Namely providing progress reports and attaining their insight and guidance to implement them on what is being conducted.

Method and approach

The discipline of computing is divided into three paradigms (Denning, 1989). These are theory, abstraction and design.

Theory is based on mathematics, and consists of the following four steps.

Characterize the objects under study.

Make hypothesis about the relationship between the objects.

Prove hypothesis true or false.

Interpret the result.

Abstraction is based on experimental scientific methods, and consists of the following steps for investigating a phenomenon.

Form a hypothesis.

Construct a model.

Design an experiment and collect data.

Analyze the result.

Design is a paradigm rooted in engineering, and consists of four steps for constructing a system that shall solve a problem.

State requirements and specifications for the system.

Design the system.

Implement the system.

Test the system.

Evaluate the system.

This thesis will follow the design paradigm, which means that a system solving a specific problem will be developed. The problem is reflected in the requirement specification. To fulfill these requirements, the system will be designed and implemented. Testing will be done to measure system performance, in our case functionality and recommender precision. Finally, the system will be evaluated by consulting test results and requirements, and then consider alternative approaches.

Outline

This thesis will consist of the following chapters:

Chapter 1 - Introductions, Background and focus area of the thesis.

Chapter 2 - Related Work introduces the information overload problem that current web technologies have to deal with. Different solutions to the problem are presented, focusing on recommender systems. The chapter ends with a case study, comparing two recommender systems that are relevant for our work.

Chapter 3 - Requirements states the system requirements.

Chapter 4 - Design proposes the design of our Music domain recommender system and its components.

Chapter 5 - Implementation discusses technical considerations and describes the implementation of the system.

Chapter 6 - Experiment describes the experiments carried out for this thesis, and the experimental results.

Chapter 6 - Evaluation evaluates the system with respect to the requirements.

Chapter 7 - Conclusion draws the conclusion of this thesis and recommends possible future work.

Research Plan

The research will be carried out over a period of 24 weeks (ref: Appendix). And will be continued in a breakdown of tasks. The end of each task is regarded as a milestone. The following is a brief detail of each task, accompanied with the expected duration to completion along with expected deliverables where it applies.

Task 1: Research (3 weeks)

Task1 will involve the studying and analysis of past and current work done the subject area. The analysis of said studies will provide a foreground to the purpose of this dissertation. A Critical review will be made on each and every study relevant to the scope of this dissertation.

Task 2: Further Investigations (6 weeks)

This task contains the activities that shall be undertaken in the form of field studies, interviews and observations.

Task 3: Design (5 weeks)

Here, all the findings of the previous tasks and the literature review will be gathered up to design and develop a guideline on what is required for a collaborative tool to be used in a web development environment. While also developing a model prototype collaborative tool will be developed with regard to the findings and guideline.

Next the design of a framework that will denote on the usage of the said tool will be drawn up as to how such a tool shall be used in the development process.

Task 4: Testing (3 Weeks)

Testing of the proposed guideline as to whether it does apply to development environments without hindrances. The testing will also conclude on how the effective the guideline is as per attesting the ideal collaboration tool required for a development environment.

This stage will also involve the testing of the prototype against the guidelines as well as its functional testing to check whether it does meet the specified standards and requirements. This may include a user test on the field or tests of limited group of expert users.

The framework as well is assessed in terms of its effectiveness on how it yields its guidance over the use of the tools to achieve the best possible results.

Task 5: Implementation (5 Weeks)

Here, the items will be implemented in either an emulated environment with set conditions and given out to work together. OR it also may be implemented in a real time development house upon agreement by both parties to give it more of a field test run, where all the elements will be used in the composing of a real web development project of a minor scope as to attain an idea of its behavior within the time frame.

Task 6: Evaluation and Conclusion (1 Week)

Lastly, the findings of all stages will be accumulated and evaluated. Reflections of the work done and results it showed on par with the amount of effort given in to it will be followed onwards.

Furthermore, the strengths and weaknesses found in the dissertation will be highlighted and listed down and explained further as to why and how they happen to exist and conditions of such elements if applies.

On conclusion, a brief glance through the overall research will be covered through while given points on future areas to explore in order to further complete the existence of the proposed study. Further time on hand will be used to improve upon the final elements of the study.

9. Deliverables

Literature Review on past and current studies of subject area.

Reports on findings on field studies done on further researchers

Guideline Design Partially working prototype

Framework Design

Implementation

Testing Specification and Criteria

Test Result Reports

Evaluation and critical analysis on the findings and observations made through the study.

Conclusion

10. Resource Required

A Desktop PC/Laptop: for the purpose of creating and maintaining documents as well as the programming of the required prototype.

Office Suite: to manage and create documentation and publishing corresponding to the research.

Web Development IDE: required to program the prototype, since it will be developed to run on the web.

My SQL: as a database management tool for the prototype application

Web Server: for the set of tasks to be done during development as well as the testing stages while it is hosted in the required environment.

Access to library resources and online journals to study areas and researches made on the scope of this research.

Participants from the general public for the surveys and testing stages.

Experts of the field to gain further insight if and when required.

Conclusion

Recommender systems are a powerful new technology for extracting additional value for a business from its user databases. These systems help users find items they want to buy from a business. Recommender systems benefit users by enabling them to find items they like. Conversely, they help the business by generating more sales. Recommender systems are rapidly becoming a crucial tool in music domain on the Web. Recommender systems are being stressed by the huge volume of user data in existing corporate databases, and will be stressed even more by the increasing volume of user data available on the Web. New technologies are needed that can dramatically improve the scalability of recommender systems.

Bibliography

Angil, A., 2008. Internet usage statistics - The big picture of world Internet users and population stats. [Online]

Available at: http://www.internetworldstats.com/stats.htm.

[Accessed 1 January 2013].

Daniel, j., 2005. Fourth Workshop on the Evaluation of Adaptive Systems in conjunction. [Online]

Available at: http://www.easy-hub.org/hub/workshops/um2005/challenge.html

[Accessed 20 Feb 2013].

Denning, J., 1989. Computing as a discipline.. [Online]

Available at: http://doi.acm.org/10.1145/63238.63239,

[Accessed 26 January 2013].

Goldberg, D., Nichols, D. & Terry, D., 1992. Using collaborative ¯ltering to weave an. ACM Press, Volume 35, p. 61.

Gulli, A. & Signorini, A., 2005. The indexable web is more than 11.5 billion pages. Japan, Proceedings of 14th International World Wide Web Conference.

Jannach, D., 2006. Finding preferred query relaxations in content-based recommenders. s.l., Proceeding of 3rd International IEEE Conference on Intelligent Systems.

Mahmood, T. & Ricci, F., 2009. Improving recommender systems with adaptive conversational. ACM Press, Volume 1, p. 73.

Mahmood, T. & Ricci, F., 2009. Improving recommender systems with adaptive conversational. Hypertext, Volume 1, p. 73.

Melville, P., Mooney, R. & Nagarajan, R., 2001. Content-boosted collaborative filtering. In Proceedings of the 2001, Volume 40.

Resnick, P., Iacovou, N., Suchak, M. & Bergstorm, P., 1994. An Open Architecture for Collaborative Filtering of Netnews.. s.l., Proceedings of ACM Conference.

Resnick, P. & Varian, H., 1997. Recommender systems. Communications of the ACM, Volume 40, p. 56.

Resnick, P. & Varian, R., 1997. Recommender systems. [Online]

Available at: http://doi.acm.org/10.1145/245108.245121

[Accessed 13 January 2013].

Ricci, F., 2002. Travel recommender systems. IEEE Intelligent Systems. ACM Press, Volume 17, p. 6.

Selberg, E., 1999. Towards Comprehensive Web Search. PhD thesis, University of Washington.