Techniques to Extract Topical Experts in Twitter: A Survey

18 Apr 2018

Disclaimer:
This essay has been written and submitted by students and is not an example of our work. Please click this link to view samples of our professional work witten by our professional essay writers. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of EssayCompany.

Techniques to Extract Topical Experts in Twitter: A Survey

  • Kuljeet Kaur, Jasminder Singh

 

ABSTRACT

An Online Social Network (OSN) such as Facebook, Twitter, Google+ etc. socially connects users around the world. Through these social media platforms, users generally form a virtual network which is based on mutual trust without any personal interaction. As more and more users are joining OSNs, the topical expert identification is a literal necessity to ensure the relevance and credibility of content provided by various users. In this paper, we have reviewed the existing techniques for extraction of topical expertise in Twitter. We provide an overview of various attributes, dataset and methods adopted for topical expertise detection and extraction.

Keywords: Topical Experts, OSN, Twitter

1. INTRODUCTION

Various OSNs allow the exchange of real-time information across wider audience in fraction of seconds. In microblogging sites like Twitter, posts may be viewed as a micropost(e.g., 140 character tweet). Also, microblogs assist users in getting their microposts reach the audience in microseconds. Likewise sensors, wherein real-time data comes in with every second, every micropost has shorter life span due to numerous posts from varied locations each second [19]. According to [11], Twitter is the new Facebook in 2014 with 288 million monthly active users [12] and 500 million tweets per day [13].

As users on OSN grow exponentially, a credible search system is must to find relevant users. In other words, how a user can rely and trust on the content he comes across in the Twitter. These credible, relevant, reliable authors or experts on a specific topic are termed as Topical Experts. Recognizing this, Twitter had launched WTF service (Who to Follow) in 2010 to extract experts related to a topic. But, it is found that WTF [21] sometimes generate results constituting of users whose Twitter profile description (called “bio”, 160 characters personal description) contain the related query but in actual, are not really related to the content.

The traditional approaches identify topical experts using attributes like tweets, profile bio, lists, number of followers etc. Our work consolidates the existing approaches and highlights future directions. The structure of the paper is organized as follows: next Section describes methodology to carry out this review. In Section 3, we explain motivation behind review. In Section 4, we highlight attributes used for various approaches, followed by Section 5, which presents comparative analysis of work done. Section 6 constitutes discussions and in Section 7, we discuss research direction. Finally, Section 8 concludes the paper.

2. METHODOLOGY

For the concerned topic, the papers included for review are selected from major databases like IEEE Xplore, ACM Digital Library, Google Scholar etc. The databases returned around 50 papers, out of which papers published after 2009 were shortlisted. Then titles and abstracts were read from the shortlisted papers. Finally, 13 papers were selected whose title and abstract were closely related to topical extraction in Twitter. The compilation of various approaches in the form of review is of prime importance to new researchers, for extension of work in this domain.

3. MOTIVATION BEHIND REVIEW

As numbers of authors in Twitter are growing exponentially; identifying influential users is of utmost importance in this era. For any user, the question for whose content to read to get updated and reliable information pertains to identification of topical expertise. So, mining such experts for any topic in order to keep in close contact with them or following them, is one of the most important domains explored by various researchers. This paper lists all the approaches, which will help researchers to review the work done in this area.

4. ATTRIBUTES USED IN VARIOUS STUDIES

From the previous studies, it can be deduced that different authors have apparently used different combination of attributes to find authoritative users on a topic. Below are the features used in categorizing and distilling topical experts in the preceeding studies:

  • Tweet[20]: 140 characters message, can be textual or can contain links to multimedia content
  • Retweet[20]: Forwarded tweets form retweets
  • Mentions[20]: Replies to the messages with @Username
  • Hashtags[20]: #topic or #keyword presents all tweets related to a topic or keyword
  • #Followers[20]: Number of users who receive your tweets in their timelines
  • #Followings[20]: Also called friends, whose tweets you receive on your timeline
  • Bio[20]: 160 characters personal description
  • Lists[20]: 140 characters name and an optional description, used for managing followers

5. COMPARATIVE ANALYSIS OF VARIOUS APPROACHES

OSN is the fastest way for disseminating real-time information. Twitter, acts both as a micro-blogging and social-networking site. Following any user in Twitter doesn’t require any access right from the person, thus circle of virtual friends grows to form the social network. Table 1 represents the previous studies’ detailed analysis regarding topical expert extraction with contribution. As, information grows exponentially with the users, thus, a credible search system is needed to find relevant users. By relevant users, we mean the experts of a topic as well as the seekers also, who help in spreading message to anonymous larger audience.

Jianshu Weng et al.[1] found influential twitterers with a specific topic. The author described that Twitter itself gives more influence to users with more no. of followers. But, focusing on only ‘following’ relationships is not reliable as the trends of following back a friend either due to courtesy or common interests (homophily) is analysed. Thus, to find the influential twitterers, TwitterRank approach is applied which considers both link structure and topic sensitivity into account. To analyze the link structure in the collected data, all friends and followers of each user were considered along with their tweets. LDA was used to mine topics of a user from tweets for topic sensitivity, followed by ranking of users’ influence. The results showed that active twitterers don’t imply influential twitterers. They either share some followers or followings of one another. The experiments showed highest similarity between this algorithm and TSPR [18] due to topic sensitivity.

Aditya Pal and Scott Counts [2] identified topical authorities on the basis of tweets, mentions and graphs using Gaussian Mixture Model clustering method. The tweets related to oil spill, iphone, world cup were mined using sample substring. Self-similarity score showed level of expertise of a user in a specific topic. The clustering algorithm was applied on 17 features, followed by ranking of authors in the 3 selected categories. The survey conducted showed that the users find tweets as useful and interesting from the top authors represented by this approach. It also showed that users trust either quality content or renowned authors presented to them. The 2 most important features concluded are topical signal (extent of involvement of an author with a topic) and mention impact (@username while replying or referring to other users).

Author

Attributes used

Methodology

Dataset

Results

JianshuWeng et.al.[1]

  • Tweets
  • Following relation
  • LDA approach
  • Graph based
  • Top 1000 Singapore based Twitterers from Twitterholic.com
  • Topic sensitive influential Twitterers tracked with improved accuracy

Aditya Pal and Scott Counts[2]

  • Tweets
  • 17 features used
  • Clustering based
  • 5 days’ tweets from firehose dataset
  • Tweets collected for the selected 3 categories seemed authoritative and informative to the users

Parantapa Bhattacharya et.al.[3]

  • Tweets
  • Lists
  • Semantic approach based on Lists, profile, tweets
  • 38.4M Twitter user’s profiles
  • Identified topical groups on niche topics, and missing member if any

HemantPurohit et.al.[4]

  • Tweets
  • Profile metadata
  • 3 methods proposed
  • Modified tf-idf approach
  • Twitter profiles
  • Wikipedia
  • Personal websites
  • US Labor statistics
  • Promised 92.8% summaries as informative in best case and 70% in worst case

Claudia Wagner et.al.[5]

  • Lists
  • Bio
  • Tweets
  • Retweets
  • LDA approach
  • Wefollow directory
  • Twitter profiles
  • Best results with Bio and Lists

Daniela Pohl et.al.[6]

  • Tweets
  • Modified tf-idf
  • Online Clustering alogorithm
  • 1943 tweets on Hurricane Sandy, 2012
  • Online Clustering algorithm with minimized clusters uncovered all subevents

Kevin R. Canini et.al.[7]

  • Tweets
  • Link Structure
  • LDA approach
  • Tf-idf approach
  • Wefollow directory
  • Twitter profiles
  • Content and social status of experts affect trust of followers

Saptarshi Ghosh et.al.[8]

  • Lists
  • Mining Lists meta-data
  • Ranking experts
  • 54M Twitter profiles
  • Better performance than the Twitter WTF service for more than 52% of the queries.

Shaomei Wu et.al.[9]

  • Lists
  • Snowball Sampling
  • Ranking experts
  • Firehose dataset of 42M users
  • Elite users are responsible for spreading the content to larger audience

Naveen Sharma et.al.[10]

  • Lists
  • Mining Lists meta-data
  • Ranking experts
  • 54M Twitter profiles
  • Cloud of attributes, describing a person’s interests is generated

Table 1. Comparative Analysis of Existing Approaches

Parantapa Bhattacharya et.al.[3] utilized lists to find topical groups (experts + seekers) and analyzed their characteristics. The study highlighted many differences between topical and bond based groups in terms of size, member type, interests etc. It is found that community detection algorithms can’t be applied due to weak connectivity between experts and seekers. From the collected data, first, experts of a topic were found followed by seekers, then merging the two to form a topical group. The approach successfully discovered niche topical groups. It is noteworthy that numbers of experts are directly proportional to numbers of seekers. The approach resulted in a single connected component covering of 90% of the experts, which shows well inter-connectivity between experts.

Hemant Purohit et.al.[4] proposed approaches to generate automatic informative summaries of users in limited characters. To generate summaries, 3 approaches were used, namely, Occupation Pattern based, Link Triangulation based and User Classification based. 92.8% of summaries generated by Link Triangulation are considered to be informative and useful on the basis of evaluation done by users, considering readability, specificity and interestingness metrics. For the users, who were less popular and active, meformer data (written by user himself, self-descriptive) was used to generate the summary. Wikipedia pages were also considered as a source of informer data for the generation of summary in Link Triangulation method, which showed highest favorability.

Claudia Wagner et.al.[5] elaborated that out of tweet, retweet, bio and List, which user-related content make a good topical expertise profile. Two experiments conducted by choosing experts with known topic of expertise from Wefollow directory. The first experiment resulted in worst expertise judgment when the participants were shown only content (tweets+retweets) which changed into best when contextual information (bio+tweet) were shown. The second experiment done to know the similarity of inferred topics from 4 user-related data, analyzed that lists performed the best by revealing 77.67% of the exact topic of interest of expertise. The similarity of topics shown by tweets and retweets is also noteworthy. Another contribution made is that bio plays an important role in inference of topic.

Daniela Pohl et.al.[6] represents implication of using social media data for emergency management. The dynamic selection of features from the incoming data using online clustering algorithm uncovered sub-events(effects of events or crisis). The terms extracted from incoming data and those with highest frequency were given maximum importance and used for clustering. The evaluation done on Hurricane Sandy, 2012, real-data showed that both online and offline clustering are similar in behavior but quality-wise online outperforms the offline clustering algorithm. Another noteworthy analysis constitutes lower set of clusters in online clustering algorithm due to ignorance of earlier sub-events.

Kevin R. Canini et.al.[7] concentrates on finding which factors do users trust more to judge the credibility of authors. The experiment showed, more quantitative the content is, more is the trust earned. Thus, content and social structure affects credibility to a great extent. Based on these 2 factors, an algorithm is proposed to find topical expertise and ranking them automatically. The comparison between the algorithm and the professionally ranking expertise’ algorithm shows great results in favor of the proposed approach.

As, lists depend on crowd wisdom, Saptarshi Ghosh et.al.[8] proposed a topical expert search system which uses single feature, Twitter Lists for inferring the topical experts. The methodology includes collection and mining of all public lists of 54 million users who joined Twitter before August 2009. The mining of meta-data generated many topics, each user was ranked according to an algorithm [17] and then association of member is done with the topic according to his rank. The membership of a user in many lists, created by many users adds certain topics to the category in which a user is an expert. Unless previous studies in the context, which uses either user’s own information(bio+tweets) or network graph to extract experts of a topic, the relying of study only on the wisdom of crowds(Twitter Lists) makes the study unique. The analysis shows that Cognos[8] provides better results as compared to the official WTF for more than 52% of the queries. Another noteworthy result came out is that WTF relies more on organizational accounts, whereas Cognos follows personal accounts to get the information, this implies not relying only on the standard news agencies but giving equal importance to each Twitter user.

Shaomei Wu et.al.[9] contributed significantly by classifying users into elite and ordinary topically, life span of content directly proportional to type of content, and how information flows indirectly to a larger audience. Lists are used for finding elite users using snowball sampling. The elite users mined for each of the 4 categories are found to be more active. The elite users besides constituting of only 0.05% of total population, 50% attention in the twitter is created by them. It is also found that textual content has shorter life-span as compared to multimedia content. The two-step flow policy highlights forwarding of elite users’ content either via retweet(acknowledged content) or reintroducing content(unacknowledged) to a wider audience.

The study by Naveen Sharma et.al.[10] is related to the previous study[14], which used a machine learning technique to find the semantic topic of a web page. D. Ramage et al.[16] used LDA to analyze the content of tweets semantically. D. Kim et al. [15] applied chi-square distribution on tweets to associate them topically. This study generated a cloud of attributes by mining the lists’ meta-data and associating the mined topics with the members of the list. Inferred attributes include information from bio, perceptions of users and topics of expertise. For checking accuracy and reliability, ground truths and human feedbacks were considered. The analysis showed 94% of the evaluations to be accurate.

6. DISCUSSIONS

The studies discussed above rely more on self-provided information (bio) and ‘following’ relationships. The generation of automatic summaries of Twitter users from tweets, bio, mentions etc. is quite subjective in nature. The validation of above studies on wider audience with varied topics may vary the results if larger sample space is considered. However, the outcomes based on analyzing social-media data may have far-reaching consequences if so applied in real world such as policy decision in government, business or any individual organization. There may be a need to test the outcome, intelligent information, on basis of certain parameters by discounting possible chances of irrelevant or disinterested information. The degree of inaccuracy, thus, has to be analyzed in each and every analytic method in mining OSN data intelligently. Only then, it will be possible to utilize the true potential of OSN for generating intelligence and situation learning.

7. RESEARCH DIRECTION

A lot of work has already been done for utilizing twitter data for various purposes. As OSN covers data from ordinary users, spammers, and experts thus extraction of useful data is needed for building intelligent recommendation systems. An important aspect for collecting credible data in the form of tweets could be via ‘Lists’ on a certain topic, which provide a way to follow a class of users, who are believed to be topical experts in a single timeline. After extracting topical expertise by mining lists’ metadata, if their tweets are analyzed semantically followed by some annotations supporting their opinion, can primarily help in predicting sensitive issues such as terrorism, riots etc. and counter-measures required to cope with them. Other useful implications of topical experts could be business forecasting, market research, financial decision-making, stage of product life-cycle, opinion-polls, crime-patterns, social-trends, econometrics and response of stake-holders to specific topics such as plebiscite or referendum etc. However, appropriate weightage may have to be accorded in the algorithm to discount misleading information campaigns by rival opinion-makers.

8. CONCLUSION

The OSN provides real-time data covering global audience and topics. To analyze data intelligently, various approaches are used, that use different attributes. From the survey, it is concluded that lists, a crowd sourced feature, if created carefully can give indications regarding topic of expertise of its members. The reason for favoring lists more than other attributes lies in its association with crowd-wisdom. Also, a list is the best way to differentiate elite users from general OSN users with crowd-sourcing as its prime feature. The other attributes, like fake bio, being provided by the user, may mislead the search system. Moreover, previous studies have proved that combination of other attributes with lists generate more accurate results.



Request Removal

If you are the real writer of this essay and no longer want to have the essay published on the our website then please click on the link below to send us request removal:

Request the removal of this essay
Get in Touch With us

Get in touch with our dedicated team to discuss about your requirements in detail. We are here to help you our best in any way. If you are unsure about what you exactly need, please complete the short enquiry form below and we will get back to you with quote as soon as possible.