Twitter User Rank Using Keyword Search

8 downloads 19766 Views 603KB Size Report
This method utilizes tweet relation among Twitter users and tweets. (retweet and reply) as well as tweet count of each Twitter user. ... Many Twitter client ..... FR gets the best result in the case of “Fukushima”, while it falls below the other.
Twitter User Rank Using Keyword Search Tomoya NORO, Fei RU, Feng XIAO, and Takehiro TOKUDA Department of Computer Science, Tokyo Institute of Technology, Japan Abstract. Twitter has attracted attention recently as a new way of collecting, providing and sharing information on the Internet. However, it is difficult for us to find Twitter users to follow for getting valuable information related to a topic of interest. In this paper, we propose a method for ranking Twitter users using the Twitter keyword search. This method utilizes tweet relation among Twitter users and tweets (retweet and reply) as well as tweet count of each Twitter user. Experimental results show our method outperforms methods using only tweet count, tweet relation, or follow relation. Keywords. Social Network Analysis, Twitter, Microblog, Keyword Search, Power Iteration Algorithm

Introduction Twitter is one of the most popular microblogging service. It facilitates real-time information collection, provision and sharing by using messages of less than 140 characters, called “tweets”. Twitter has 140 million active Twitter users and delivers 340 million tweets a day in March 2012 [1], and the number of tweets delivered in a day is still growing. In Twitter, many users (we refer to “Twitter user” as “user” in this paper) post tweets about various topics. Some of the tweets are very interesting although they are not provided by the major media. Not all users posts such tweets about all topics equally, but experts and related people in each topic provide such information and their own opinion. For example, in the case of electronic books, some writers, editors, publishers, lawyers and so on will provide a lot of valuable information and opinion from their own view point. Also, some users play a role of curator by collecting information related to the topic they are interested in from various sources and posting them. If we follow such users, we can easily get such valuable information related to the topic we are interested in. However, it is difficult for us to find such users in some intuitive ways. For example, if we would like to find users posting interesting tweets about electronic books, it is not enough for us to see how many tweets about electronic books they post. We need to take into consideration how much influence they have by watching user relation. What is “user relation” in Twitter? There are two types of user relation: follow relation and tweet relation. Follow relation is built by following other users and tweet relation is built by retweeting/replying to other users’ tweets or mentioning other users 1 . 1 Although, in general, difference between “mention” and “reply” is not clear, we refer tweets including user names (strings preceded by “@”) as “mention tweets”, and tweets targeted to particular tweets as “reply tweets”. “Mention” is used in the more general sense than “reply” in this paper.

Twittomender [2] uses follow relation to find users related to a particular user or query. TwitterRank [3] also uses follow relation to find influential users. However, we think follow relation is not suitable for finding users to follow. We can freely follow other users 2 and easily get many followers by randomly following many non-followers and waiting for them to follow back. Follow relation built in this way is meaningless since most of the followers may not pay attention to our tweets and we have little influence on them (and we may not be interested in their tweets either). Even if we follow users in a different way, we do not actually pay attention to all of their tweets especially in the case that the number of friends 3 and the number of their tweets are too large. Many Twitter client applications, such as TweetDeck, Echofon, HootSuite, YoruFukurou and Twipple 4 , have been developed and some of them can sort our friends’ tweets into some groups. Twitter also has an official feature called “lists” [5] and we can create some user groups to follow separately from following each user. Some users search for tweets by hashtags[6] and trends[7]. As a result, following users does not always mean we actually see their tweets and following relation does not always reflect user influence. On the other hand, we think tweet relation reflects user influence more clearly since retweeting/replying to a user’s tweet or mentioning a user means that we actually see the tweet or the user and take interest in the information or the user’s activity. Cha et al. [8] shows that users who have many followers are popular but not necessarily influential, while users who are retweeted and mentioned many times have ability to post valuable tweets and ability to engage others in conversation respectively. It means that observing retweet, reply and mention is more appropriate for finding users to follow for getting valuable information related to the topic we are interested in. Although Cha et al. observe only retweet and mention count, we think relation built on such user activity (retweet, mention and reply) is also useful. In this paper, we present a method for ranking users using the Twitter keyword search, i.e. finding users to follow for getting valuable information related to a search query. This method utilizes not only tweet count of each user but also tweet relation among users and tweets by calculating “tweet count score” and “user influence score”. Experimental results show that our method outperforms methods using only tweet count, tweet relation, or follow relation. The organization of the rest of this paper is as follows. In section 1, we describe overview of our method. After we explain how to collect tweet data in section 2, we present calculation of tweet count score and user influence score in section 3 and 4. Evaluation results are shown in section 5, then we discuss some related works in section 6. Lastly we conclude this paper in section 7.

1. Overview of Twitter User Rank Using Keyword Search In our method, given an input query representing the topic of interest, we first get tweets matching the query using the Twitter keyword search, then extract user information and 2 Every user can follow 2000 users, and there are limits to the number of additional following. This limit is based on the user’s ratio of followers to following [4]. 3 Although some call a user we follow “followee”, we refer to it as “friend” in this paper. 4 TweetDeck: http://www.tweetdeck.com/, Echofon: http://www.echofon.com/, HootSuite: http://hootsuite.com/, YoruFukurou: http://sites.google.com/site/yorufukurou/home-en, Twipple: http://twipple.jp/

'(%&) !"#$$%&./%+&,-

!"#$$%&

*%+&,-.&%/(0$. 1$"%%$/2

3/%&.4+5%. %6$&+,$#74

3/%&. &+4;#4< !"%%$.&%0+$#74. %6$&+,$#74

3/%&.#4879. +,:(#/#$#74

Figure 1. Flow of TURKEYS Calculation

tweet relation from them. After we create a reference graph based on the tweet relation, we calculate Twitter User Rank using KEYword Search (TURKEYS) of each user (Figure 1). We assume the followings in this paper. 1. Users who post many valuable tweets and retweets about the topic are worth following. 2. Valuable tweets attract attention from many users. 3. Each user pays attention to tweets he/she retweets or replies to. Also, he/she watches the other tweets to some extent (not more than retweets and reply tweets). Based on these assumptions, we define the TURKEYS score of each user u as follows. TURKEYS(u) = TC(u)w × UI(u)1−w

(1)

TC(u) and UI(u) are respectively tweet count score and user influence score of user u ranging between 0 and 1 (details are described later). Weight w also ranges between 0 and 1. The tweet count score is based on tweet count of each user and reflects the first assumption. Users who post many tweets related to the topic will be ranked higher. However, if we consider only this assumption, some users who repeatedly posts similar tweets like advertisement will also be ranked higher. Such users should be excluded since their tweets are not valuable. Then we consider the user influence score, which is based on tweet relation among users and reflects the second and the third assumptions. Users who posts many worthless tweets will be ignored by others, in other words, their tweets are neither retweeted nor replied to. As a result, such users will be ranked lower since the user influence score is low.

2. User And Tweet Data Collection Each tweet has many attributes, some of which are listed below. id: The tweet ID user: Information of the tweet poster including the poster’s name, the number of friends and followers, profile, and so on.

',-.)/+0-&1-2&3&42-'5

!!

!"

!#

!$

!%

&'()*+',)

Figure 2. Reply Chains of Tweets

text: The tweet text in_reply_to_status_id: (In the case of a reply tweet) tweet ID the tweet replies to in_reply_to_screen_name: (In the case of a reply tweet) user name the tweet replies to retweeted_status: (In the case of a retweet) information of the original tweet (full information of the original tweet is embedded) We use these attributes for calculating the TURKEYS score. We collect tweets and users as follows. 1. Given an input query representing the topic of interest, get at most recent 1,000 tweets matching the query by the Twitter search API [9]. Let this tweet set be T0 . 2. Get detail information of each tweet in T0 by the Twitter API statuses/show/id, since the Twitter search API does not return full information of each tweet and some information such as retweeted status is not included. 3. Get tweets replying to each tweet in T0 by the Twitter API related_results/show/ id. Let this tweet set be Tr . Tr = {t|t 0 ∈ T0 ∧ t 0 .id = t.in_reply_to_status_id}

(2)

where t 0 .id indicates ID of the tweet t 0 and t.in_reply_to_status_id indicates tweet ID replied to by the tweet t. The Twitter API related_results/show/id returns not only tweets replying to the target tweet but also tweets in reply chains including the target tweet (tweets replied to by the target tweet, tweets replying to the tweet replying to the target tweet, and so on) 5 . In this step, we get only tweets whose attribute in_reply_to_status_id directly points a tweet in T0 . Figure 2 shows an example of reply chains of tweets. Dashed arrows indicate in_reply_to_status_id relation. If we send ID of the tweet t1 ∈ T0 to the API related_results/show/id, a set of tweets {t0 , t2 , t3 , t4 } is returned. We get t2 and t3 , which t1 is directly pointed by. 4. For each tweet in T0 ∪ Tr , get the poster’s name, the tweet ID, tweet ID and user name the tweet replies to (in the case of a reply tweet), tweet ID and poster name of the retweeted status (in the case of a retweet), and user names in the tweet text (strings preceded by “@”). Let the set of tweets obtained and the set of users obtained be Tall and Uall respectively. 5 The Twitter API related_results/show/id is not officially supported by Twitter and at most 8 tweets can be obtained.

3. Tweet Count Score The tweet count score of each user is simple. It is calculated by counting not only original tweets but also retweets in T0 as each user’s own tweets. Users who retweets many tweets posted by others may also be good users to follow since they play a role of “filter” searching for valuable tweets related to the topic and sharing them with their followers. The score is normalized so that the largest value should be 1 (i.e. tweet count of each user is divided by tweet count of the user who posted the most tweets and retweets). TC(u) is represented as follows. TC(u) =

|{t|t ∈ T0 ∧ t.user.id = u.id}| maxu 0 ∈Uall |{t|t ∈ T0 ∧ t.user.id = u 0 .id}|

(3)

where t.user.id indicates poster’s ID of the tweet t and u.id indicates ID of the user u.

4. User Influence Score We present two versions of user influence score calculation. Difference between the two versions is how to create reference graphs of the tweet relation. One version is based on a reference graph consisting of only user nodes, and the other version is based on a reference graph consisting of user nodes and tweet nodes. 4.1. User-Only Version of User Influence Score Calculation This version is based on PageRank [10], whose idea fits the assumption of Twitter users to follow presented in section 1. In this version, we create a user reference graph from the tweet set Tall consisting of user nodes and directed edges among user nodes. It represents who retweets/replies to whose tweet and who mentions whom. We use Au to denote an adjacency matrix of the graph. Au (u i , u j ) = retweet(u i , u j ) + mention(u i , u j )

(4)

where u i and u j indicates the i-th user and the j-th user respectively (1 ≤ i, j ≤ |Uall |). retweet(u i , u j ) is the number of times u i retweets u j ’s tweet and mention(u i , u j ) is the number of times u i mentions u j (u i replies to u j ’s tweet). The adjacency matrix Au are transformed as follows. { Au (u i ,u j ) ∑ ∑ (1 − d) + |Udall | if k Au (u i , u k ) 6= 0 k Au (u i ,u k ) (5) Bu (u i , u j ) = 1 otherwise |Uall | where d is a damping factor of 0 ≤ d ≤ 1. This transformation reflects the third assumption of twitter users to follow described in section 1. Each user pay attention to users he/she actually retweeted or mentioned. Also, he/she watches all users (regardless of his/her activity of retweet and mention) at a certain rate of “d”. The user influence score is calculated as follows. u = BuT u

(6)

u0 = ( |U1all | , |U1all | , . . . , |U1all | ); k = 1; Repeat uk = BuT uk−1 ; k = k + 1; until |uk − uk−1 | < ε; return uk ; Figure 3. Calculation Algorithm of the User Influence Score

%&''%

34'*

!!

"! 34'*

!"

"$

!#

%&''% ""

!$

"#

%&''%'()*'%&''%'(+,-

./-+/%%'0%120+%2

Figure 4. Tweet-User Relation

where u indicates the column vector of the user influence score of all users. We can calculate the user influence score using the power iteration method like PageRank [10], HITS [11] and SALSA [12]. The iterative processes are as follows. uk = BuT uk−1

(7)

where uk indicates the user influence score at the k-th iteration respectively. Calculation algorithm of the user influence score is shown in Figure 3. ε is error tolerance. Lastly the user influence score of each user is normalized so that the largest value should be 1. UI(u j ) =

u( j) maxk u(k)

(8)

The final solution represented in Eq. (1) is calculated using Eqs. (3) and (8). 4.2. User-Tweet Version of User Influence Score Calculation This version is an extension of the user-only version. In this version, we define not only the user influence score of each user but also tweet influence score of each tweet. Roughly speaking, users with high user influence score post many tweets and retweets, and tweets with high tweet influence score are paid attention to by many users (Figure 4). Based on this idea, the user influence score of each user is calculated using the tweet influence score of original tweets and retweets the user posted, and the tweet influence score of each tweet is calculated using the user influence score of users who pay attention to the tweet. We create a tweet-user reference graph from the tweet set Tall consisting of user nodes, tweet nodes, and directed edges between a user node and a tweet node. We use

At and Ar to denote adjacency matrices of the graph. At represents what (tweet) is posted/retweeted by whom (user) and is used for calculating the user influence score, and Ar represents who retweets/replies to what and is used for calculating the tweet influence score. Note that rows and columns of At respectively indicate tweets and users while those of Ar have opposite meanings.   if u j tweets ti 1 At (ti , u j ) = wr if u j retweets ti posted by another user   0 otherwise { 1 if u j retweets or replies to ti Ar (u j , ti ) = 0 otherwise

(9)

(10)

where ti and u j indicates the i-th tweet and the j-th user respectively (1 ≤ i ≤ |Tall | and 1 ≤ j ≤ |Uall |). wr is a value of 0 ≤ wr ≤ 1. If wr is 1, the original poster and retweeters are equally treated as posters of the tweet. If wr is 0, retweeters are ignored. If wr is between 0 and 1, although retweeters are treated as posters of the tweet, the original poster takes priority to retweeters. Some tweets include user names (strings preceded by “@”) but do not have information of the target tweets (in_reply_to_status_id and in_reply_to_screen_name). In this case, we search for the target tweets. If the target tweets are not found, we add virtual tweet nodes to the tweet-user reference graph. The details are described in appendix A. Adjacency matrices At and Ar are transformed as follows. At (ti , u j ) Bt (ti , u j ) = ∑ k At (ti , u k ) { Ar (u j ,ti ) ∑ (1 − d) + k Ar (u j ,tk ) Br (u j , ti ) = 1

(11) d |Tall |

|Tall |

if

∑ k

Ar (u j , tk ) 6= 0

otherwise

(12)

where d is a damping factor of 0 ≤ d ≤ 1. At ∑ is transformed just by dividing each element by the corresponding row sum. Note that k At (ti , u k ) 6= 0 since any tweet is posted by at least one user. Like the user-only version of user influence score calculation, transformation of Ar reflects the third assumption of twitter users to follow described in section 1. Each user pay attention to tweets he/she retweeted or replied to. Also, he/she watches all tweets (regardless of his/her activity of retweet and reply) at a certain rate of “d”. The user influence score and the tweet influence score are calculated as follows. u = BtT t

(13)

t = BrT u

(14)

where u and t indicate the column vector of the user influence score of all users and the column vector of the tweet influence score of all tweets respectively. We can calculate the user influence score and the tweet influence score using the power iteration method like the user-only version. The iterative processes are as follows.

u0 = ( |U1all | , |U1all | , . . . , |U1all | );

t0 = ( |T1all | , |T1all | , . . . , |T1all | ); k = 1; Repeat uk = BtT BrT uk−1 ; tk = BrT BtT tk−1 ; k = k + 1; until |uk − uk−1 | < εu and |tk − tk−1 | < εt ; return uk and tk ; Figure 5. Calculation Algorithm of the User Influence Score and the Tweet Influence Score

uk = BtT BrT uk−1

(15)

tk = BrT BtT tk−1

(16)

where uk and tk indicate the user influence score and the tweet influence score at the k-th iteration respectively. Calculation algorithm of the user influence score and the tweet influence score is shown in Figure 5. εu and εt are error tolerance. Lastly the user influence score of each user is normalized so that the largest value should be 1. UI(u j ) =

u( j) maxk u(k)

(17)

The final solution represented in Eq. (1) is calculated using Eqs. (3) and (17).

5. Evaluation 5.1. Keywords Used In order to evaluate our method, we collected tweets including the following four Japanese words in Japanese characters, “Fukushima”, “electronic book (e-book)”, “fair trade” and “whaling” (in this paper, we describe the words in English so that readers can understand), and two hashtags, “#dkyo” and “#bijp”. The hashtag “#dkyo” is used for users who discuss developing and utilizing digital tools for education, and the hashtag “#bijp” is used in tweets related to basic income. We used words in Japanese characters to collect tweets in only one language. If tweets in several languages are mixed up, it is difficult to evaluate our method since interaction among users tweeting in different languages is not active and we cannot get enough user relation. Tweets in several languages (English, Portuguese, Spanish and other European languages) may be obtained if we use words in roman alphabets, since roman alphabets are used in many languages (foreign words in roman alphabets often appear even in Japanese tweets). Although we can specify language parameter in the Twitter search API, it does not always work properly (some tweets in other languages are included in the search result). Using words in Japanese characters is more likely to get tweets in only one language (Japanese) than using words in roman alphabets. Since Japanese is the second most used language in Twitter and 14% of tweets posted in October 2011 are Japanese [13], we expect that enough tweets can be obtained in the Twitter search even if we restrict the target language to Japanese.

Table 1. The Number of Tweets and Users Keyword Fukushima e-book fair trade whaling

|T0 | (% of total)

|Tall | Reply

Mention

|Uall |

Total

Retweet

1,000 999

573 (57.3%) 171 (17.1%)

46 ( 4.6%) 93 ( 9.3%)

68 ( 6.8%) 103 (10.3%)

1,280 (1,229) 1,267 (1,199)

1,073 888

630 991

157 (24.9%) 246 (24.8%)

48 ( 7.6%) 184 (18.6%)

42 ( 6.7%) 54 ( 5.4%)

701 ( 674) 1,262 (1,236)

440 837

#dkyo

97

26 (26.8%)

8 ( 8.2%)

37 (38.1%)

#bijp

642

229 (35.7%)

66 (10.3%)

170 (26.5%)

(exc. virtual tweets)

114 (

99)

36

790 ( 687)

229

Table 1 shows the number of tweets and users obtained for each keyword. |T0 | is the number of tweets obtained by the Twitter search in the first and the second steps of the data collection. “Retweet” and “reply” indicate the number of retweets and the number of reply tweets respectively. “Mention” indicates the number of mention tweets except reply tweets (tweets including user names but not specifying its target tweet). |Tall | and |Uall | are the number of tweet nodes and user nodes making up the user reference graph and the tweet-user reference graph. Numbers in brackets indicate the number of tweet nodes excluding virtual nodes. In the case of “e-book” and “whaling”, although we get 1,000 tweets in the first step of the data collection, we cannot get the details of some of the tweets in the second step due to privacy setting, tweet deletion and account suspension. Private tweets, deleted tweets, tweets posted by suspended (or deleted) users do not usually appear in search results. However, such tweets sometimes appear due to inconsistency of database for the Twitter search and complete database of all tweets 6 . From Table 1, we can see percentage of retweets in the search result of “Fukushima” is larger than the other keywords. On the other hand, percentage of reply tweets and mention tweets in the search result of “#dkyo” and “#bijp” is large. It means that, in the case of “Fukushima”, users usually tweet independently and some users just retweet interesting tweets while, in the case of “#dkyo” and “#bijp”, some users post their own opinions quoting other users’ tweets and engage other users in an active conversation. 5.2. Methods Compared with and Evaluation Metric We compare our method with other methods described below. Tweet count score (TC): Rank users based on the tweet count score described in section 3. User influence score (UI): Rank users based on the user influence score described in section 4. Follow relation (FR): Rank users based on follow relation among users using PageRank [10]. We create a follow relation graph (consisting of user nodes) from friend lists of all users in Uall 7 . Adjacency matrix of the follow relation graph is represented as follows. 6 Database for the Twitter search is different from complete database of all tweets. Only tweets posted in 6-9 days are stored in the Twitter search database. We think that inconsistency will occur if change of privacy setting, tweet deletion, and so on has just been executed. 7 We need to get the friend lists after the last step of the user and tweet data collection described in section 2.

Table 2. Relevance Score between User and Keyword Score

Description

2

The user often posts tweets related to the input keyword

1 0

The user sometimes posts tweets related to the input keyword The user does not usually post tweets related to the input keyword The user posts much more tweets not related to the input keyword

{ A f (u i , u j ) = { B f (u i , u j ) =

1 if u i follows u j 0 otherwise

(18)

A (u ,u ) ∑ f i j (1 − d) + d |Uall | k A f (u i ,u k ) 1 |Uall |

if

∑ k

A f (u i , u k ) 6= 0

otherwise

(19)

where u i and u j indicates the i-th user and the j-th user respectively, and d is a damping factor, which is set to 0.15. Evaluation is done on the top-n users ranked by each method with respect to discounted cumulative gain (DCG) [14]. DCG is defiend as follows. DCGn = rel1 +

n ∑ reli log2 i

(20)

i=2

where n is the number of users we evaluate and reli indicates relevance score between the user ranked i-th and the input keyword. Definition of the relevance score is described in Table 2. Users who continuously post tweets related to the input keyword are good users to follow since we usually expect users to post interesting tweets in the future when we follow them 8 . Based on this idea, we manually assign relevance scores to top-20 users ranked by each method about a month after we carry out user ranking in order to see whether users continuously post tweets related to the input keyword. Users who only post almost the same tweets (e.g. advertisement) or meaningless tweets (e.g. just listing the Twitter trend words) repeatedly are assigned relevance score of 0. Users posting too many tweets not related to the input keyword are also assigned relevance score of 0 even if they sometimes posts related tweets since we will see many tweets of no interest if we follow such users. In the case that some users get the same score in each method but are assigned different relevance scores, we sort the users in descending order of relevance score. We normalize DCG score by dividing the score by the maximum value since DCG score gets higher with increasing value of n. maxDCGn = 2 +

n ∑ i=2

nDCGn =

2 log2 i

DCGn maxDCGn

(21)

(22)

8 If we would like to see the current tweets related to the input keyword, we just search for tweets using the Twitter search and we do not have to decide who to follow. It is not the purpose of this research.

#$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

47"7'

!"#$(

!"#$)

*+

47"7%#

,-

./

!"#

%$47"70#

!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version #$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$) 47"7%#

*+

,47"70#

(c) User-tweet version (wr = 0.5)

./

!"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$) 47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 6. nDCG Score of Each Method for “Fukushima”

5.3. Result Our method is evaluated changing w in Eq. (1) and wr in Eq. (9). Parameter d in Eqs. (5) and (12) is set to 0.15. Normalized DCG (nDCG) score at n = 5, 10, and 20 of each method for each keyword is shown in Figures 6, 7, 8, 9, 10, and 11. Although TC and FR do not depend on the version of UI (and its parameters), they are shown in all of the graphs for comparison. Except for the case of “Fukushima”, we can see that our method gets better result than the other methods, especially in the case of n = 5 (i.e. evaluation on the top-5 users). It means that our method ranks good users to follow higher compared to the other methods. Our method gets better results than TC and UI since our method can exclude users who repeatedly post similar tweets like advertisement (users with high TC score) and users who post few tweets but are retweeted and replied to many times (users with high UI score) by combining the two measures. Although the value of w at the best nDCG score differs among keywords, our method achieves better result on average in the case of w = 0.5, which means TC and UI should be evenly weighted for our task. We cannot see significant difference in the value of wr , which tells us whether retweeters should be differentiated from the original tweet poster does not greatly affect the user influence score. Our method using the user-only version of UI calculation seems better than our method using the user-tweet version except for the case of “whaling”, although

#$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

!"#

%$47"70#

!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version %

#$3

#$3

#$2

#$2

45+6

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$)

47"7%#

*+

,47"70#

(c) User-tweet version (wr = 0.5)

./

!"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$) 47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 7. nDCG Score of Each Method for “electronic book”

the user-only version just considers who retweets/replies to whose tweet while the usertweet version considers who posts/retweets what as well. Some important information might be lost during the power iteration (Eq. (15)) due to transformation of the tweetuser reference graph into “user-only” reference graph, although we have not found what the lost information is as of yet. FR gets the best result in the case of “Fukushima”, while it falls below the other methods in the other cases. We use 1,000 tweets including each keyword for this evaluation. Since too many tweets including “Fukushima” are posted (after the earthquake in Japan), we can get 1,000 tweets including “Fukushima” posted in only 30 minutes. We cannot get enough tweet relation from the tweets due to the short time period and our method and UI do not work well. The result of TC is not so good either since not so many users post many tweets in only 30 minutes and there is not so much of difference in tweet count of each user. Many users are interested in topics of “Fukushima” (the nuclear plant accident) and most of the users know who to follow to get information of the topic. As a result, follow relation would be better to search for users to follow to get information of very popular topics. On the other hand, FR gets the worst result except for the case of “Fukushima”. It means that follow relation is not efficient for searching for good users to follow. As we mentioned at the beginning of this paper, follow relation do not always reflect user influence since we have a variety of ways to see tweets posted by other users besides

#$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

!"#

%$47"70#

!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version #$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

47"7'

!"#$(

!"#$)

47"7%#

*+

,47"70#

(c) User-tweet version (wr = 0.5)

./

!"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$) 47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 8. nDCG Score of Each Method for “fair trade”

following such as the Twitter search, the Twitter lists, the Twitter trends, and original functions of some Twitter client applications. nDCG score of FR is much lower than the other methods in the case of “#bijp” since users who often use the hashtag communicate with other users searching for tweets by the hashtag. nDCG score of FR is not so bad in the case of “#dkyo” since most of the relevant users actually follow one another.

6. Related Work Twitter provides its original user search service. We can search for users by clicking the link “People” in the Twitter search result page. It mainly shows users whose screen names or profiles match the input keywords and does not care whether they actually post tweets related to the keywords. As a result, it shows some users who do not usually post related tweets. Although popular keywords such as “Fukushima” may work well since everyone often posts related tweets, many users who do not post any tweets related to the keywords recently are shown in the search result in the case of not so popular keywords. In contrast, our method shows users who actually post tweets including the keywords at least when the search is done. Taking user profile information into consideration may improve our method, and it is left to future work.

%

#$3

#$3

#$2

#$2

45+6

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

47"7'

!"#$(

!"#$)

*+

47"7%#

,-

!"#

%$./

!"#$&

!"#$'

!"#$(

47"7'

47"70#

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version #$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$) 47"7%#

*+

,47"70#

(c) User-tweet version (wr = 0.5)

./

!"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$) 47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 9. nDCG Score of Each Method for “whaling”

Twittomender [2] uses lists of followers, friends, and terms in their tweets to find users related to a particular user or query. As we mentioned in section 5.3, follow relation is not efficient for searching for good users to follow. Our method uses tweet relation instead and we showed that our method outperforms the method using follow relation except for the case of very popular keywords. TwitterRank [3] also uses follow relation to find influential users. It is an extension of PageRank algorithm and takes both topical similarity among users and link structure (follow relation) into account. Although purpose of TwitterRank is not exactly the same as ours, it could also search for users based on keywords by determining the topic corresponding to the keyword and measuring the influence of each user in the topic. However, as shown in our evaluation result, tweet relation is more efficient for searching for good users to follow than follow relation. Cha et al. [8] shows that users who have many followers are popular but not necessarily influential, while users who are retweeted and mentioned (replied to) many times have ability to post valuable tweets and ability to engage others in a conversation respectively. Our evaluation result supports this claim. However, they just watch the number of retweets and mentions to measure user influence. We think that tweet relation (who retweets or replies to whose tweet) is important to determine user influence, and this idea is incorporated into our method.

#$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

!"#

%$47"70#

!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version #$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$) 47"7%#

*+

,-

./

47"70#

(c) User-tweet version (wr = 0.5)

!"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$)

47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 10. nDCG Score of Each Method for “#dkyo”

7. Conclusion In this paper, we presented a method for ranking users using the Twitter keyword search to find users to follow for getting valuable information related to the search keywords. Our method utilizes tweet count of each user and tweet relation among users and tweets. We showed that our method outperforms methods using only tweet count, tweet relation, or follow relation except for the case of very popular keywords such as “Fukushima”. Although we evaluated our method by giving only one word as its input, the number of input words is not limited to one. Our method can accept any types of query that the Twitter search can deal with (e.g. AND/OR/NOT operations, phrases, location) since our method uses the Twitter search. If the input query can be created by obtaining characteristic words in some news articles, we could search users to get valuable information related to the news topic. We could use our method not only for user recommendation but also for arranging the Twitter search result by emphasizing tweets posted by high-ranked users or ignoring tweets posted by low-ranked users. In the future, we need to improve our method for the case of very popular keywords. One solution could be getting more tweets to rank users. Twitter provides the streaming API [15] to get tweets in real time. If we are allowed to take much time to return ranking results (e.g. 1 day, 3 days, or 7 days), it could be possible to get enough tweets by using the streaming API.

#$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

!"#

%$47"70#

!"#$&

!"#$'

!"#$(

47"7'

!"#$)

*+

47"7%#

,-

./

47"70#

(b) User-tweet version (wr = 0.0)

(a) User-only version #$3

#$3

#$2

#$2 45+6

%

45+6

%

#$1

#$1

#$0

#$0

#

# !"#

%$!"#$& 47"7'

!"#$'

!"#$(

!"#$)

47"7%#

*+

,47"70#

(c) User-tweet version (wr = 0.5)

./

!"#

%$!"#$&

!"#$'

!"#$(

47"7'

!"#$) 47"7%#

*+

,-

./

47"70#

(d) User-tweet version (wr = 1.0)

Figure 11. nDCG Score of Each Method for “#bijp”

Our method does not directly take into consideration whether each user continuously posts tweets related to the input keyword. It may be solved by watching temporal change of user rank although it is also a time-consuming task since we need to check user rank regularly. Although we showed follow relation is not efficient for searching for good users to follow, considering follow relation sometimes works. For example, users who do not have so many followers but are followed by many of the users interested in particular topic may be good users to follow for getting information related to the topic. Such users will have high FR score. We would like to think of new method incorporating follow relation into our method to find such users. Lastly, at this point, we do not use tweet text and user profile for user ranking. Considering words in tweet text and user profile will improve our method.

References [1] [2]

Twitter. Twitter turns six. http://blog.twitter.com/2012/03/twitter-turns-six.html. John Hannon, Mike Bennett, and Barry Smyth. Recommending Twitter users to follow using content and collaborative filtering approaches. In 4th ACM Conference on Recommender Systems (RecSys ’10), pages 199–206, 2010.

[3]

[4] [5] [6] [7] [8]

[9] [10] [11] [12] [13] [14] [15]

Jianshu Weng, Ee-Peng Lim, Jing Jiang, and Qi He. TwitterRank: Finding topic-sensitive influential Twitterers. In 3rd ACM International Conference on Web Search and Data Mining, pages 261–270, 2010. Twitter. Following rules and best practices. http://support.twitter.com/articles/68916-following-rulesand-best-practices. Twitter. How to use Twitter lists. http://support.twitter.com/articles/76460-how-to-use-twitter-lists. Twitter. What are hashtags ("#" symbols)? http://support.twitter.com/articles/49309-what-are-hashtagssymbols. Twitter. About trending topics. http://support.twitter.com/articles/101125-about-trending-topics. Meeyoung Cha, Hamed Haddadi, Fabrício Benevenuto, and Krishna P. Gummadi. Measuring user influence in Twitter: The million follower fallacy. In 4th International AAAI Conference on Weblogs and Social Media, 2010, 2010. Twitter. Using the Twitter Search API. https://dev.twitter.com/docs/using-search. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford University, 1998. Jon M. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5):604– 632, 1999. Ronny Lempel and Shlomo Moran. The stochastic approach for link structure analysis (SALSA) and the TKC effect. Computer Networks, 33(1-6):387–401, 2000. Semiocast. Arabic highest growth on twitter – english expression stabilizes below 40%. http://semiocast.com/publications/2011_11_24_Arabic_highest_growth_on_Twitter. Kalervo Jarvelin and Jaana Kekalainen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422–446, 2002. Twitter. Streaming API. https://dev.twitter.com/docs/streaming-api.

A. How to Deal with Mention Tweets in Creating a Tweet-User Reference Graph Some tweets include user names (strings preceded by “@”) but do not have information of the target tweets (in_reply_to_status_id and in_reply_to_screen_name). In the step of creating a tweet-user reference graph for calculating the user-tweet version of the UI score, such tweets (we refer to them as “mention tweets”) are processed as follows. 1. If the mention tweet tm is a retweet with a comment, such as “Yup! RT @username: Prayer works!” 9 , we get the following part of “RT @username: ” (i.e. “Prayer works!”) and search for the original tweet similar to the text among tweets posted by “username” in T0 . If the original tweet to is found, we let the mention tweet tm be a reply tweet to the original tweet to . 2. Otherwise, we add a virtual tweet node tv to the tweet-user graph and let the mention tweet tm be a reply tweet to the virtual tweet tv . We assume that the user appearing in the mention tweet tm posted the virtual tweet tv . If more than one user appear in the mention tweet tm , we assume the virtual tweet tv is posted by all of the users, i.e. At (tv , u 0 ) = 1 for each user u 0 in the mention tweet tm . The tweet similarity calculation in the first step is based on edit distance. sim(tm , to ) = 1 −

edit_distance(to , tm .tail) |tm .tail|

(23)

where tm .tail indicates the following part of “RT @username: ” in the mention tweet tm , edit_distance(to , tm .tail) indicates edit distance from to to tm .tail, and |tm .tail| indicates 9 Not only “RT” but also “QT” are accepted. The colon preceded by “RT @username” is not necessary.

length of tm .tail. In the case of a retweet with a comment, some parts of the original tweet may be deleted due to limitation of tweet length. When we calculate edit distance from (a candidate of) the original tweet to the mention tweet (retweet with a comment) we focus on, we do not count the delete cost while we set both the replace cost and the insert cost to 1. We get a tweet with the highest similarity score and, if the similarity score is higher than a threshold 0.5 10 , we take the tweet as the original tweet of the mention tweet we focus on.

10 Some may think that the threshold should be set higher. However, some users add some words (or hashtags) at the end of the original tweet when they post retweets with comments. we set the threshold lower to deal with the situation. In our preliminary experiment, we can get original tweets in 100% precision.