Web Pattern Analysis In Modern Web

Published Date: 02 Nov 2017

The web has become an increasingly popular medium for consumer to exchange or find ideas, opinions, experiences on products and services. Many consumers go further than online information sharing and actually perform purchases on the web. Recently several domain-specific systems have been developed addressing the problem of authenticity and correctness of the data available on the web example; recognize reliable comment in products review forums etc. Major drawback of most research efforts is the lack of a general framework applicable to all online forum and usage. The Web prediction problem (WPP) can be generalized and applied in many essential industrial applications such as search engines, caching systems, recommendation systems, and wire- less applications. Statistical analysis leads to the discovery of existence of diverse patterns of human information sharing activity in dissimilar online social networking sites and forums. This discovery is employed as prior knowledge in the classification framework, which decompose the original highly imbalanced problem into several balanced sub-problems. Therefore, it is crucial to look for scalable and practical solutions that improve both training and prediction processes. In this research, the aim to provide consumers with a choice of using a web content recommender system that generates personalized recommendations in a timely manner based on a model of their own habits and behaviors.

 Introduction:

As the scale of the Internet are getting larger and larger in recent years, we are forced to spend much time to select necessary information from large amount of web pages created every day. To solve this problem, many web page recommender systems are constructed which automatically selects and recommends web pages suitable for user’s favour. Though various kinds of Web Pages have been constructed,

there are many points to be improved in them. Most of past web page recommender systems uses collaborative filtering. Collaborative filtering is often used in general product recommender systems, and consists following two stages.

1. Analyse users purchase histories and extract user groups which have similar purchase patterns

2. Recommend products which are commonly preferred in the user’s group

The recommender system may be classified into three broad categories. They are:

Personalized recommendation - recommend things based on the individual's past behaviour

Social recommendation - recommend things based on the past behaviour of similar users

Item recommendation - recommend things based on the item itself

A combination of the three approaches above

Related Work:

Web Content Recommender System based on Consumer Behavior Modeling:

Web mining is used to understand the users' frequent access patterns and behaviors. The recommender system generates recommendations provide customers with choice. In this approach the access pattern is mined periodically during the frequently accessed times. For example, morning 8 to 10 people watch about the traffic news and weather report and evening about entertainment. The data is collected from the users logs which are semantically enhanced. This is done by adding a tag to the requested URL of the user. The tag will contain the semantic information of the URL such as sports, traffic, entertainment, reality, etc. This task can be done manually or semi-automatically. The tokenized URL consists of the timestamp which gives information about what type of information the user requested during what times of the day. Based on that consumer behavior knowledge base can be generated. The predefined attributes for representing a website can be used as the tags. The tags may have sub categories too. To build consumer based knowledge base the logs are preprocessed and fuzzy periodic web uage content is extracted. From the knowledge of information about other similar global users the knowledge base is constructed.

Extracting Patterns and Relations from the World Wide Web:

This paper discusses about the earlier concept of Big data in which a specific useful information is extracted from a large set of data. The amount of information available on the World Wide Web is large and it is mostly unstructured and in different formats ranging from gene databases to medical transcripts to list of spas. If such data are integrated into a structured form it'll generate a massive and more complex source of information. The pattern will have a particular format of occurrences of records or values that are present in the dataset and not out of it. Since the databases are diverse the pattern may have various representaions.

Let p be a pattern. Then MD(p) is the set of tuples that match p in D and |p|D is the number of elements in MD(p).

Web usage, advertising and shopping: relationship pattern

Another way of classification of the patterns is based on the web usage level of the users. The web users are classified into three categories based on the time they spend using web. They are high, medium and low web users. It is assumed that high web users are more sophisticated and well to do so they spend more time in web. So the recommedation system for them for providing item details for shopping would prove worthwhile but that is not the case with medium and low web users. The medium and low web users doesnt know about web advetising so they possibly ignore it but the high web users have more chance of looking into the web for additional details and looking in the web.

Real-time Recommendation and pattern recognition system around us

Social networking sites such as Linkedin provides "people you may know" suggestions by matching the data we give such as the school we went to, the employer we worked for. Sometime it gives the results based on the skill sets of the people too.

Goodreads suggests the books to read based on our previous read books and the genres we most often read. This framework also recommends semi-automatically where the people who are friends to us can recommend books for us to read.

Facebook provides suggested sites and pages which might catch our interest. The new graph search is purely based on the pattern matching where the pages visited by the similar users are suggested to us and inturn we can find people based on their interest which we specify.

Survey:

Here, we try to understand various methods that may be used by a recommender system.

User Profiles:

A profile of the user’s interests is used by most recommendation systems. This profile may consist of a number of different types of information. Here, we concentrate on two types of information:

A model of the user’s preferences, i.e., a description of the types of items that interest the user. There are many possible alternative representations of this description, but one common representation is a function that for any item predicts the likelihood that the user is interested in that item. For efficiency purposes, this function may be used to retrieve the n items most likely to be of interest to the user.

A history of the user’s interactions with the recommendation system. This may include storing the items that a user has viewed together with other information about the user’s interaction, (e.g., whether the user has purchased the item or a rating that the user has given the item). Other types of history include saving queries typed by the user (e.g., that a user searched for an Italian restaurant in the 90210 zip code).

There are several uses of the history of user interactions. First, the system can simply display recently visited items to facilitate the user returning to these items. Second, the system can filter out from a recommendation system an item that the user has already purchased or read. Another important use of the history in content-based recommendation systems is to serve as training data for a machine learning algorithm that creates a user model.

The training data of a classification learner is divided into categories, e.g., the binary categories "items the user likes" and "items the user doesn’t like." This is accomplished either through explicit feedback in which the user rates items via some interface for collecting feedback or implicitly by observing the user’s interactions with items. For example, if a user purchases an item, that is a sign that the user likes the item, while if the user purchases and returns the item that is a sign that the user doesn’t like the item.

Given a new item and the user model, the function predicts whether the user would be interested in the item. Many of the classification learning algorithms create a function that will provide an estimate of the probability that a user will like an unseen item. This probability may be used to sort a list of recommendations. Alternatively, an algorithm may create a function that directly predicts a numeric value such as the degree of interest.

Personalized Web Content Recommendation:

Personalized web content recommendation aims at minimizing ambiguity and unwanted information that is presented to the consumer, thereby reducing the effect of information overload that is often encountered by web surfers. According to a survey presented in, traditional web content recommender systems could be classified into Content-based, Collaborative and Hybrid, which is combination of the two. However, these systems tend to rely heavily on user ratings. Non-intrusiveness has been identified

as an important attribute in the information gathering process for subsequent web content recommendation. Web usage mining, which is performed by the system in the background

and transparent to the user, therefore represents an important way forward in this area of research.

The fundamental requirement of an effective personalized web content recommendation system is to present the most relevant suggestions to the user in a timely manner. Thus, both context and temporal information is important. In this regard, context information can be very effective in disambiguation.

For example, a biologist who wants to read articles about "mouse" will likely be interested in the rodent. On the other hand, a consumer looking for computer accessories will likely be interested in the pointing device. Context information will help resolve this type of ambiguity.

Periodicity-Based Pattern Mining:

Discovering periodic patterns from time series databases is an important data mining task for many applications. According to the type of patterns, periodic patterns can be divided into periodic association rules and periodic sequential patterns.

For web usage mining, association rules can be used to find correlations between web pages (or products and services on an e-commerce website) accessed together during a session. Such rules indicate the possible relationships between pages that are often viewed together even if they are not directly

connected, and can reveal associations between groups of users with specific interests. Apart from being exploited for business applications, the association rules can also be used for web recommender systems, web personalization, or improving the system's performance through predicting and pre-fetching of web data.

Periodic association rules are rules that associate with a set of events that occur periodically. Such association rules hold only during certain time intervals but not others. Calendar information is critical in describing the time intervals.

Periodic sequential pattern mining can be viewed as an extension of sequential pattern mining by taking into account the periodic characteristics in the time series data. Obviously, traditional sequential pattern mining techniques can be applied to find periodic sequential patterns.