2016 IEEE International Conference on Big Data (Big Data)
Leveraging Social Big Data for Performance Evaluation of E-Commerce Websites Eyad Makki, Lin-Ching Chang Department of Electrical Engineering and Computer Science The Catholic University of America, Washington, DC, 20064, USA
[email protected],
[email protected] Abstract— E-commerce plays a key role in business success nowadays. Therefore, the performance of E-commerce websites is critical. E-commerce websites generate a large amount of data that is often used for performance evaluation. Many website evaluation methods have been proposed, but the social media factor is usually not taken into consideration. In this paper, Twitter data is utilized for big data analytics and several Twitter performance indexes are proposed to assist the website performance evaluation. The result is compared with the performance evaluation result using several commercial tools for 13 selected E-commerce websites in Saudi Arabia. Keywords- E-Commerce, Social Media, Big Data Analysis, Website Evaluation, Saudi Arabia
I.
INTRODUCTION
E-commerce websites collect a huge amount of data from various sources such as system, network, customers, and social media. Big Data Analysis (BDA) is adopted by Ecommerce businesses handling both structured data, like demographic data, and unstructured data, like traffic, in order to be ahead of their competitors. One BDA application is the performance improvement for E-commerce business. For example, it was used to generate personalized recommendations, incentivize purchases with special offers, and streamline the checkout process for a faster, better, and more intimate shopping experience [1]. Social data has also been utilized in E-commerce businesses. Social media plays a critical role in E-commerce development; it increases the ability to reach a larger group of customers online. There are many benefits to be gained from social media supporting E-commerce. Social media helps in promoting brand awareness, generates more business and richer user experiences that was confirmed by many studies [2]–[6]. It also provides the ability to rapidly spread ads, and broaden targeting customers by the followers. It is an effective way to communicate with customers and receive their feedback. In fact, social media helps to drive recommendation that leads the hesitancy customers to purchase, thanks to the increased degree of confidence in the product or service, as well as the reliance on the trusted sources with respect of word of mouth. Feedbacks of customers and rating of products can be easily obtained through their comments, likes and reviews, which facilitate interactive ways and relationships development. From technical perspective, social media supports Search Engine Optimization due to the positive relationship between higher search rankings and wider markets and customers. In practical E-commerce, social media provides marketing insights reflected by the increased customer loyalty through building trust with products or services and mainly with the company brand itself [7].
978-1-4673-9005-7/16/$31.00 ©2016 IEEE
2525
There are many traditional, simple approaches used for E-commerce websites evaluation. These approaches are based on: checklist methods, knowledge experience, regression model, and analog approach. They can solve problems based on a set of systematic steps ignoring the associations between global decision making process factors. Therefore, some significant factors influencing the outcomes are the ability and the experience of the analyst [8]. The effectiveness of E-commerce websites is crucially important as it directly impacts the compatibility, usability, revenue and growth of the business. Qualitative approach using principals or criteria were often used to evaluate the overall quality of an E-commerce site. However, the qualitative evaluation performed by a domain expert can be subjective and time-consuming [9]. Moreover, the Ecommerce environment is not static and data is collected all the times, so an automatic quantitative approach to continuously evaluate the performance of a site would be highly desired. A number of websites evaluation methods using quantitative approach have been proposed; each method was designed with one or a few particular evaluation purposes. For examples, some methods are designed for general websites and some are customized for specific domains [10], while others are used for redesign decisions or for strategic decisions related to business or marketing [9], [10]. Researchers or field engineers usually extended an existing foundation model like WebQual [11] or ServQual [12] by adding new criteria and dimensions to meet their own evaluation purposes [13]. In addition, these tools usually use a set of pre-defined factors without providing a way to take other important factors into account. These existing tools often report only a set of parameters such as the speed, accessibility, link analysis and page ranking that may not be the most effective indictors for the growth of the business. Moreover, these existing approaches fail to adequately incorporate social media factor. The use of social media plays an important role in online purchase decision. For example, Makki and Chang reported that social media affects the purchase decision for female Saudis [14]. The use of social media helps in building a stronger customer relationship and in increasing the product awareness. The number of social media accounts and their massive usage in Saudi Arabia also indicate consumers’ readiness to employ social media on E-commerce. Twitter is one of the largest social media platforms in the world and is the most popular one in the Arabic world. Twitter Search API provides a way to evaluate E-commerce sites through measuring social engagement, translated in the post and reply rates; user interaction reflected in retweet rates, favorite rate, and direct messages; and response rate or
speed. At the time of writing, Twitter has 320 million users, which makes it the sixth largest social networking site in the world. As Jansen and Zhang pointed out [15], no other social media platform allows a direct and instant communication between consumers and businesses, and free APIs to streaming data. The importance of Twitter also emanates from the fact that tweets are not only displayed on a user’s profile page, but they can be delivered directly to followers via instant messaging, Short Message Service, Really Simple Syndication, email, or other social networking platforms. One discrepancy among countries regarding Twitter usage is clear if we calculate the Twitter penetration for each country. The penetration is defined as the number of monthly active tweeting users relative to the total amount of internet users in that country. Statistical reports and information graphics about social media usage in Saudi Arabia puts it in top of other countries in the region and the world with over 32% active twitter users [16]. Saudi Arabia’s Twitter growth rate is of 3,000% from 2011 to 2012, and is still growing. This growing rate is 10 times more compared to the average global rate. Statistics also showed that there was an average of 50 million tweets per month in 2012 and 150 million tweets in 2013 in the country [5]. The afore mentioned numbers made Saudi Arabia ranked number one in the world in Twitter growth and activity, which motivate us to use the social media data from Twitter to assist the evaluation of Ecommerce websites in Saudi Arabia. Many recent phenomena and new factors affecting Ecommerce performance are globally recognized which include but are not limited to social media, mobile usage, payment method, and government regulation. It is also a known fact that the factors influencing E-commerce adoption and implementation are highly related to the country or the region and its culture under investigation [17]–[20]. These new factors, region and culture differences needed to be considered in a modern E-commerce websites evaluation model [21]. Thus, a new tool based on such model that better tackles fore mentioned problems is indeed needed. Therefore, our ultimate goal is to develop an Online Presence Evaluation Model (OLPEM) for performance evaluation of E-commerce sites. In this paper, however, we do not intend to create such new tool but focus on social media and region factors and leverage them for performance evaluation. The region under study is Saudi Arabia and the social media channel is Twitter. From a previously collected data of over 160 E-commerce websites in Saudi Arabia [5], we chose the top 13 Twitter followers accounts that have web traffic data. Tweets data in a 28-day period was collected to analyze the performance of E-commerce sites through factors like popularity, engagement (response) and reach. II.
LITERATURE REVIEW
A. Evaluation of E-commerce Sites Evaluation of the performance of websites and assessment of the impact is important to the successfulness of any E-commerce business. It is also a way to define their strengths and weaknesses in order to enhance the former and address the latter. Website evaluation allows business owners
2526
and investors to measure the site’s success and deficiency. Consumers will also benefit from the evaluation results as it will provide them with the information they need regarding E-commerce services and product quality as well as the business’ overall reputation. In fact, consumers as well as Ecommerce owners have been using third party comparison websites to have a neutral, reliable evaluation of business websites. Comparison sites are considered as one of the most reliable methods for E-commerce evaluation today as they provide an impartial assessment based on a set of predefined criteria. For instance, BizRate.com, one of the very first comparison websites, established 10 criteria to evaluate Ecommerce websites [9]. Comparison sites bridge the gap between E-commerce websites and their consumers. So, customers have a way to navigate the growing landscape of online retailers to assess the quality of different online stores. Other methods that have been used to evaluate Ecommerce websites are online surveys, expert evaluation, or the combination of both. For instance, Forrester Research is an independent technology and market research company which combines online surveys and experts analysis to come up with a comprehensive evaluation. Gomez Advisors Company is famous for what its experts call “scorecard,” which encompasses criteria comprising 150 to 250 factors to measure the quality of E-commerce websites. Shihab et al [22] have taken safety as a new factor and created a new formula to measure safety when shopping online. They call it the “Trust Presence Evaluation” tool which would include a larger number of parameters that they obtained from other 10 similar studies. They reported that lack of trust in Ecommerce transactions may potentially hinder further growth of E-commerce. Their study was the first one that focused on the level of trust displayed by sellers online. They formulated a trust presence evaluation tool that includes 9 categories, 29 parameters, and 93 indicators to be used to assess the trust presence in E-commerce websites. Creating and maintaining a website is one of the most important branding tools to improve the brand’s visibility. Social media has a growing and an undeniable impact on any businesses nowadays. According to Weatman [23], building an online presence in social media is vital for a brand to thrive and to be acknowledged by consumers. She stressed on the fact that with their omnipresence, social media represents a huge opportunity for all kinds of E-commerce to get the publicity it needs and to reach hundreds of millions of customers. Weatman provided advice to business owners on how to build the online presence through various social media, including Facebook, Twitter, LinkedIn, Pinterest, and YouTube. Her paper suggested that businesses need to grow a positive online presence and ensure the development and promotion of a polished, well-represented brand image. To evaluate an E-commerce online presence, it is important, as the businesses’ fan-base grows, to use analytics and other tools to help understand how effective an online presence is, and how to make it even more effective, such as knowing where or what specific social media a business needs to focus its efforts on and what they might need to change. Talpau et al [24] have enumerated the benefits of Ecommerce online presence, namely; overcoming
geographical boundaries, increasing speed of communications, 24/7 access to information, low cost promotion, the possibility of establishing inter-personal relationships, the ability to segment the market very well, the ease in obtaining feedback, and high exposure. Their study, as they qualify it, is descriptive and exploratory. The data is collected from public sources and then analyzed to identify Romanian top brands behaviors in an online environment. Exploratory studies are used to understand a problem or a situation. Descriptive studies help building behavioral profiles of people, events, and situations and in this case the brands. The authors explained online presence in terms of the brand strength and weakness; they proved that the presence of brands in the social media and brand strength are interdependent. Social media helps brands to interact with customers, using humanizing strategies of the brand which can lead to the customers’ loyalty. Clarifying the difference between website and social media presence, the authors noted that the brand's experience in the online environment is the site and the contact with the brand through social media, while in the office environment, the brand is represented more by products attributes. More than with the website, social media has the potential to create relationships with customers, interact with them and retain them, which is why brands must be active in the social media in order to establish contact with the target audience. Companies must focus on trust and the first step is to create brand loyalty [24]. B. E-Commerce in Saudi Arabia AlGhamdi et al [17] presented a new conceptual framework aiming to promote trust in E-commerce environment in Saudi Arabia. The new model is a result of mixed methods conducted in four stages involved in the study to determine the reasons of the slow growth of Ecommerce in Saudi Arabia. The framework is a five-part model aiming to promote trust in E-commerce in Saudi Arabia. These five parts are: the provision of secure and trustworthy online payment alternatives, enforcing consumer protection, strengthening delivery systems, clarifying market restrictions, and establishing certification authority. The results of the study confirmed that the disbelief in Ecommerce environment in Saudi Arabia is the main reason of the slow growth. Alotaibi and Bach introduced an exploration of the major challenges to establish efficient facilities faced by the Ecommerce industry in Saudi Arabia [25]. These challenges of the successfulness of E-commerce implementation in Saudi Arabia are: the absence of government involvement, insecure online payment infrastructure, the lack of E-commerce law, and weakness of the postal delivery system. Therefore, some proposed solutions were presented to address challenges facing the process of E-commerce development facilitation in Saudi Arabia. Their solutions require the support of government to establish permanent home addresses and to provide secure online payment, as well as to establish regulations and policies. Makki and Chang [26] agreed on the challenges and solutions but reported several new factors. Other web evaluation studies [27] have reported that a few
2527
factors such as heuristic and usability are no longer affecting the performance of websites, which explains why these factors will be excluded in this research. C. E-Commerce Website and Social Media Evaluation in Saudi Arabia Alotaibi proposed an evaluation instrument aiming to evaluate the quality of E-commerce websites in Saudi Arabia and then highlight the potential improvement opportunities [10]. Consequently, six E-commerce websites have been chosen for analysis and evaluation, equally categorized as follows: regional, international, and domestic. These websites are: Logta, E-Mall, Souq, Amazon, E-Market, and Alibaba. Sixty participants have evaluated those six websites, following a number of evaluation instrument guidelines with good web design experience. Evaluation factors described by the evaluation instrument are: content, interaction, assurance, appearance, organization, and customer focus. The outcomes of the instrument have confirmed that the regional and domestic websites were of lower quality than international websites considering all web evaluation aspects. Therefore, the quality of domestic websites is less than the quality of regional websites in terms of customers’ focus, appearance, organization, assurance and content. Several hypothesis were formed by Guzzo et al [28] regarding how the consumer’s purchase decision is affected by other members of social network with the same social attraction and interests. Hence, a new model called Ecommerce Consumer Acceptance (ECA) was developed taking into account the additional components that affect consumer acceptance with respect to online social networks. Their new model can be considered as an extension to the Technology Acceptance Model formalized from the social network point of view to determine the influence of social media on consumer’s acceptance of E-commerce. The study added a better understanding of the online social effects on the E-commerce acceptance. Therefore, their model extends other variables associated with purchase decision affected by a set of characteristics of social networks. Alshehri and Meziane put their attention to the influence of social media as a third party on E-commerce development especially in building trust measures [3]. They described that close social relationships, mainly family members and friends, has stronger impact of recommendations on the purchase decision in Saudi Arabia. They also attempted to explore whether Saudi Arabia’s case is similar to the findings in other studies. More specifically, their study evaluated the effects of social endorsements on E-commerce transactions to determine the role of social impact in the increase of online shopping in the country. They performed a survey and collected data from 606 participants in Saudi Arabia. Based on their survey’s results, they reported four main factors associated with the social characteristics and relationships affecting online shopping. In details, a strong effective factor was a member of a famous group of companies that improves the trust in online shopping with 83.3%, followed by 81.5% of the participants trust in their family members. In addition, the percentage of the
participants that trust their friends’ recommendations was 73.8%. In general, 57.1% of the respondents are fine with supporting factors of social impact as a third party that enhances online shopping. The findings of the study have confirmed that the similarity between previous studies conducted in other countries and Saudi Arabia. Moreover, they reported that the social influence is a motivating factor, which supports and strengthens E-commerce development in Saudi Arabia. Alafeef identified the influence of social media on Ecommerce sales based on a sample selected E-stores from Saudi Arabia [7]. He reported a positive relationship between the use of social media and E-commerce sales and promotions in the conducted companies. Thus, he suggested the use of social media as a changing tool to the development of E-commerce in Saudi Arabia. He stated that E-commerce is one of the current beneficial methods to enhance business operations in terms of increasing purchase and marketing efficiency in Saudi Arabia. Makki and Chang also conducted a similar study related to E-Commerce and social media in Saudi Arabia [14]. D. Twitter for Analytic Purposes Twitter analytics has been extensively used among researches seeking to collect huge amount of data and find useful information on different subject fields. Jamison-Powel et al, used twitter data to conduct a research on insomnia [29]; others analyzed twitter data for marketing [30] or athletics purposes [31], just to name a few. Other studies using Twitter data for analytics cover a wide range of topics including but not limited to information propagation and diffusion [32], popularity prediction [33], event detection [34], and tweet classification [35]. Most of these studies focused on qualitative content analysis, linguistic analysis or network analysis; some studies focused on quantitative analysis by employing statistical method and algorithm development like ours. III.
METHODOLOGY
In this section, we describe the data used in this paper, how and where the data was collected, and what factors were extracted from Twitter data. We propose several Twitter performance indexes and explain how we use these indexes for E-commerce site performance evaluation. We then compare the twitter performance indexes with the performance evaluation result using several commercial tools for 13 selected E-commerce websites in Saudi Arabia. A. Data Collection Methods In this study, we used a quantitative approach to gather an in-depth understanding of Twitter usage from Ecommerce websites in Saudi Arabia. The top 13 Twitter followers accounts were selected from our previous study containing data of more than 160 Saudi E-commerce sites [5]. As pointed out [36], social media analytics has played an important role in E-commerce. In this study, we aim to leverage the Twitter data to assist the performance assessment for E-commerce systems. To accomplish our goal, we collected data from online Twitter analysis
2528
websites, TweetChup.com and Twitonomy.com. In addition to Twitter data, we also collected website traffic, ranking, regional related and services data for each selected website. The dynamic website created in our previous study (ecgulf.org) containing data of E-stores, like category, location, website link, social media accounts links, delivery methods, accepted payment methods, languages of website, currencies accepted, mobile compatibility etc. [5]. More than 160 major online stores in Saudi Arabia were included in ecgulf.org. Information of each E-store was manually added first by visiting its corresponding website and filling record form in our site. E-store owners can also visit the site and fill the form to be included in our database. The actual number of E-stores in our site can vary as we regularly add new E-stores or remove the out-of-business E-stores. At the time of this paper is written, the number of E-stores in our website is 164. We also collected the overall web traffic data the of 13 selected E-stores from SimilarWeb.com. In summary, Twitter data, website traffic and regional related data were collected to formulate their influences on performance, and to evaluate how they affect the overall E-store performance as part of our development of OLPEM. Twitter Data Collection: Tweets of each account from June 1st, 2016 to June 28th, 2016 are gathered using Tweetchup.com and our Twitter Search API web program. The total number of tweets collected during this 28-day period is 667,391. This Twitter dataset is analyzed using an in-house web-based program to extract useful information. The following information is extracted for each account: account creation date, number of followers, times the account is added to a list by followers and total lifetime tweets. Table 1 shows the selected accounts and summary of their data. Table 1: Twitter Accounts Data Summary Twitter Account
Created
Followers
Listed
Total Tweets
alhabib_shop
11/18/2012
76,480
135
7,918
alwaneshop
10/02/2012
60,548
110
41,871
axiom_ksa
07/05/2012
195,992
274
51,745
Carrefoursaudi
05/21/2011
295,958
349
3,701
dokkanafkar
08/08/2012
44,134
87
8,503
EMall_KSA
06/23/2010
86,664
261
28,105
eXtraStores
09/08/2009
616,234
778
63,296
jarirbookstore
04/13/2011
1,336,817
1,679
252,408
MarkaVIP
09/07/2010
54,895
135
19,786
matjarhk
07/15/2012
63,594
97
4,280
NamshiDotCom 11/17/2011
313,948
327
111,027
SouqKSA
01/18/2010
284,253
603
53,988
vanillaeshop
09/13/2011
77,099
229
20,763
ar-sa.namshi.com, 4: carrefourksa.com, 5: dokkanafkar.com, 6: e-mall.com.sa, 7: extrastores.com, 8: jarir.com, 9: ksa.axiomtelecom.com, 10: markavip.com, 11: matjarhk.com, 12: saudi.souq.com, 13: vanillaeshop.com.
We also extracted selected time period data: number of period tweets, average daily tweets, number of mentions, number of tweets containing links, number of replies, number of retweets, number of times tweets are favorited (liked) and number of hashtags. Table 2 shows retrieved data counts for each Twitter account during the 28-day period.
Region Related and Services Data Collection: We have collected regional related and services data that include payment methods, delivery methods, aftersales services, regional coverage, website languages and accepted currencies. For each factor, we applied a pre-defined weighting scheme based on our previous studies [14], [37]. For examples, credit card payment method is desired and weights more than other payment methods. The payment score would be ranged from 1 to 10 with the following weights: Credit Card=4, PayPal=3, Bank Transfer=1.5, Cash On Delivery=1 and CashU=0.5). CashU is a payment gateway service for the Arab world. For the delivery methods, the use of more reliable local and commercial services would be preferred, so the weights is 8 for Aramex, FedEx/SMSA and Saudi Postal, and 2 if own delivery systems were used. For regional coverage, if an E-store covers Saudi Arabia, an initial score of 7 is given, and the score will be reduced by 1 if it covers limited areas in the country. If the E-store also covers the neighboring Gulf countries, Arab countries or international countries, the score will be increased by 1 for each. The main website language is Arabic which is scored of 5, and any additional languages (either one or more) will make it 10. Similarly, the main accepted currency is Saudi Arabian Riyals (SAR) which is scored 9, and any additional currencies would be scored as 10. Finally, aftersales services are counted by 2 for each service (exchange, return, maintenance, guarantee and technical support).
Website Traffic and Ranks Data Collection: We also collected website traffic data of each E-store from SimilarWeb.com for the same period of time. The traffic data collected includes: total number of visits, number of average daily visits, average visitor duration, number of pages per visit, and bounce rate. SimilarWeb.com also provides other data like traffic source, display advertising, content and audience interests. However, such information is not always available for all selected websites. Therefore, we decided not to consider it until it is available in the future. Additional data extracted is Google Page Rank, Alexa Rank, Moz Rank and Moz Domain Authorization. These ranks provide a general view on the web pages in either search engines based on a set of criteria or in their own directory using their own algorithms for a certain purpose. For example, Page Rank works by counting the number and quality of links to a page according to keywords to determine a rough estimate of how important the website is. In addition, we also checked the following information for each site including Native Mobile App availability, Google Mobile Friendly (GMF) status, and Google Safe Browsing (GSB). Table 3 shows the traffic data of the 13 E-commerce websites retrieved from SimilarWeb.com. For space-saving purpose, we will use the E-store numbers instead of the full address of the E-commerce sites in tables. The mapping is 1: alhabibshop.com, 2: alwaneshop.com, 3:
Table 2: Twitter Accounts Tweets Counts from 1, June 2016 to 28, June 2016 Average Daily Mentions Tweets 2.8 41
Twitter Account
Tweets
alhabib_shop
76
alwaneshop
226
8.4
axiom_ksa
2,751
Carrefoursaudi dokkanafkar
Links
Replies
Retweets
Favorited
Hashtags
4
37
10,229
1,086
11
80
41
32
16,961
1,513
111
101.9
2,943
627
2,657
247
430
99
140
5.2
117
31
97
626
1,797
76
69
2.6
63
12
56
17
78
0
EMall_KSA
410
15.2
215
243
186
1,229
965
964
eXtraStores
2,853
105.7
2,684
1,061
2,696
1,777
2,259
124
jarirbookstore
4,935
176.3
2,980
1,072
4,708
19,481
8,892
290
MarkaVIP
422
15.6
413
62
381
661
1,159
95
matjarhk
350
13
223
215
158
40,826
2,397
337
NamshiDotCom
1,271
47.1
1,222
71
1,238
77
227
105
SouqKSA
2,082
77
1,807
401
1,805
506
1,406
215
vanillaeshop
144
5.3
98
44
94
559
153
34
2529
Table 3: E-commerce Websites Traffic Data from SimilarWeb.com from 1, June 2016 to 28, June 2016
2,444
Average Daily Visits 87
Average Visitor Duration 00:04:50
2
4,772
170
3
756,829
4
E-store No
Total Visits
1
Pages Bounce per Visit Rate 7.62
16.57%
00:06:03
3.58
29.91%
27,030
00:07:12
10.08
41.91%
240,623
8,594
00:03:34
5.77
34.15%
5
243,715
8,704
00:07:16
8.37
33.32%
6
107,681
3,846
00:04:43
6.24
28.84%
7
2,659,492
94,982
00:05:35
5.48
31.10%
8
2,517,003
89,893
00:05:58
5.16
29.97%
9
99,373
3,549
00:03:39
4.04
35.66%
10
651,633
23,273
00:06:33
8.73
40.97%
11
9,710
347
00:04:09
4.38
28.27%
00:04:30
4.30
51.90%
00:09:02
4.41
29.01%
12 13
11,977,050 427,752 10,746
384
, where Y is the age of the account in years, T is the total number of tweets, F is the number of followers, and L is the number of listed. Tweets Activity (TA): The TA score depends on the average daily tweets reflecting how often the account tweets. The TA score can be calculated according to the following formula also using the PRE function: 𝑇𝑇
𝑇𝑇𝑇𝑇 = 𝑃𝑃𝑃𝑃𝑃𝑃 � � × 100 𝑃𝑃
, where T is the number of tweets within a period and P is the number of days in the same period. Account Reach (AR): The AR score depends on the total number of tweets and the number of followers that reflects how many tweets have been sent to how many followers. We recognize that the number of retweets may contribute to the score of this index, but due to the difficulty to track the followers of each follower who retweeted, we have decided not to include it here but use it in our next index. The AR score can be calculated using the following formula using the PRE function: 𝐴𝐴𝐴𝐴 = 𝑃𝑃𝑃𝑃𝑃𝑃(𝑇𝑇 × 𝐹𝐹) × 100
The numbers of total lifetime tweets by each account illustrates that the level of tweets is not proportional to the number of followers, see Table 1. For example, Carrefour Saudi is ranked 4th in number of followers with nearly 300 thousand while it is ranked last (13th) with only 3,701 lifetime tweets. On the other hand, alwaneshop is ranked 11th in number of followers, over 60 thousand, but is ranked 6th with over 41 thousand tweets. More details on the results will be presented and discussed in the next section.
, where T is the number of tweets within a period and F is the number of followers.
B. Twitter Performance Indexes To measure the performance of Twitter accounts, as a part of social media factor in the OLPEM, the following indexes were proposed and computed using the collected data. The score of each index presented here is ranged from 0 to 100. In general, a higher score indicates a better performance.
4
Account Age (AA): The AA score depends on the account creation date, when the account was added to a user list (listed) per follower and the lifetime tweets. The score of this index reflects how long the account being opened, how many times the account being listed, and how many tweets the account have been posted during the account’s lifetime. A Microsoft’s Excel function PERCENTRANK.EXC (PRE), which returns the rank of a value in a data set as a percentage, is used to calculate the percentile of one account compared to the average of all accounts. The AA score was calculated according to the following formula: 𝐴𝐴𝐴𝐴 = �
𝑌𝑌
𝐹𝐹
𝑃𝑃𝑃𝑃𝑃𝑃�𝑇𝑇�+𝑃𝑃𝑃𝑃𝑃𝑃� 𝐿𝐿 � 2
� × 100
Account Popularity (AP): The AP score depends on the number of tweets, the retweets percentages, the favorited percentages and the number of followers. The AP score reflects how popular is the account, and can be calculated by the following formula: 𝐴𝐴𝐴𝐴 = �
𝑇𝑇 𝑅𝑅
𝑇𝑇 𝑉𝑉
𝑃𝑃𝑃𝑃𝑃𝑃(𝑇𝑇×𝐹𝐹)+ 𝑃𝑃𝑃𝑃𝑃𝑃� �+𝑃𝑃𝑃𝑃𝑃𝑃� �+𝑃𝑃𝑃𝑃𝑃𝑃(𝐹𝐹)
, where T is the period tweets, F is the number of followers, R is the retweet percentage and V is the favorited percentage. Account Engagement (AE): The AE score depends on the percentage of replies and mentions that reflects how many times out of tweets an account replied or mentioned a follower during the observed period. The AE score can be calculated according to the following formula: 𝐴𝐴𝐴𝐴 = �
𝑇𝑇
𝑇𝑇
𝑃𝑃𝑃𝑃𝑃𝑃�𝑌𝑌�+𝑃𝑃𝑃𝑃𝑃𝑃�𝑀𝑀�+𝑃𝑃𝑃𝑃𝑃𝑃(𝐹𝐹) 3
� × 100
, where T is the period tweets, Y is the number of replies, M is the number of mentions and F is the number of followers. Account Website Support (WS): The WS score depends on the percentage of tweets containing hyperlinks within a period. The WS score can be calculated according to the following formula: 𝑇𝑇
WS= 𝑃𝑃𝑃𝑃𝑃𝑃 � � × 100 𝑈𝑈
2530
� × 100
, where T is the period tweets and U is the number of tweets containing hyperlinks. C. Website Performance Indexes Three different website performance indexes will be used to represent the performance evaluation of Ecommerce website and compare against the Twitter performance indexes. As mentioned in section III.A, the data used here was collected from SimilarWeb.com (see Table 3). The score of these three indexes are also ranged from 0 to 100. Website Traffic (WT): The WT score depends on the total number of visits (V), the average daily visits (D), the average visitor duration in seconds (R), the number of pages per visit (P) and the bounce rate (B). Intuitively, these factors are not equally important. Based on our previous studies and our best knowledge, we have defined a relative weighting scheme for these factors. The WT score can then be calculated according to the following formula with the weights pre-defined: 𝑊𝑊𝑊𝑊 = �
(𝑃𝑃𝑃𝑃𝑃𝑃(𝑉𝑉)×6)+(𝑃𝑃𝑃𝑃𝑃𝑃(𝐷𝐷)×2)+(𝑃𝑃𝑃𝑃𝑃𝑃(𝑅𝑅)×0.75)+ 𝑃𝑃𝑃𝑃𝑃𝑃(𝑃𝑃)+(𝑃𝑃𝑃𝑃𝑃𝑃(𝐵𝐵)×0.25) � 10
× 100
Website Rankings and Mobile Friendly (RM): The RM score depends on the Google Page Rank (G), the Alexa rank (A), the Moz rank (M), the Moz domain authority rank (D), the native mobile app (P) and GMF (F). If there is a native mobile app, P equals to 1, otherwise 0. If the website passed the GMF test, F equals to 1, otherwise 0. The RM score can be computed as the average value of those 6 parameters according to the following formula: 𝑅𝑅𝑅𝑅 = �
𝑃𝑃𝑃𝑃𝑃𝑃(𝐺𝐺) + 𝑃𝑃𝑃𝑃𝑃𝑃(𝐴𝐴) + 𝑃𝑃𝑃𝑃𝑃𝑃(𝑀𝑀) + 𝑃𝑃𝑃𝑃𝑃𝑃(𝐷𝐷) + 𝑃𝑃 + 𝐹𝐹 � × 100 6
Regional and Services (RS): The RS score depends on the payment methods (P), the delivery methods (D), the
regional coverage (R), the aftersales services (S), the website languages (L), and the accepted currencies (C). Similarly, these factors won’t be equally important, and relative weight should be applied to each factor. Therefore, we also defined a weighting scheme for these factors. The RS index can then be computed using the following formula:
IV.
𝑅𝑅𝑅𝑅 = (𝑃𝑃 × 3.5) + (𝐷𝐷 × 2) + (𝑅𝑅 × 1) + (𝑆𝑆 × 2.5) + (𝐿𝐿 × 0.5) + (𝐶𝐶 × 0.5)
RESULTS AND DISCUSSION Table 4 lists the scores of computed performance indexes from the collected Twitter data. More investigation is needed to determine which index is more effective for the performance evaluation of E-commerce sites. We believe a linear combination of these indexes would be an optimal choice. For the purpose of simplicity, we compute an average Twitter performance score that includes all proposed indexes, i.e., a linear combination of these indexes with equal weights. In Table 5, we show the values of computed rank using various commercial evaluation tools. We manually checked each E-commerce site to determine the existence of a native mobile application. Google Page Rank indicates how a website is performing in their search engine using Google’s own algorithm. This rank refers to the order of the website showing in searching results and the value is out of 10. Moz Rank and Moz Domain Authority have values out of 10 and 100 respectively. Unlike Google Page Rank targeting keywords, Moz Rank targets link popularity score based on search queries. This also explains the difference in range between them and will lead us to normalize the values when combining results. Table 6 listed the information needed to compute the RS index, which is a score related to regional and services. Each factor was computed for each E-commerce site based on the data we have obtained from our dynamic webpage (ecgulf.org), and applied the predefined weighting schemes we have described in the method section.
Table 4: Scores of Twitter Accounts Performance Indexes Twitter Account
AA Score
TA Score
AR Score
AP Score
AE Score
WS Score
Average Score
alhabib_shop alwaneshop axiom_ksa Carrefoursaudi dokkanafkar EMall_KSA eXtraStores jarirbookstore MarkaVIP matjarhk NamshiDotCom SouqKSA
32.10 49.95 57.10 57.10 21.40 46.40 82.10 74.95 28.55 67.80 67.80 46.35
14.20 35.70 85.70 21.40 4.10 50.00 92.80 64.20 57.10 42.80 71.40 78.50
14.20 28.50 71.40 57.10 7.10 50.00 92.80 85.70 42.80 35.70 64.20 78.50
17.80 26.75 74.95 42.80 39.25 48.20 76.75 58.90 37.45 24.95 80.30 73.15
52.33 71.37 38.03 45.20 35.67 76.13 49.97 35.67 33.30 71.40 30.90 54.73
7.10 28.50 78.50 21.40 14.20 64.20 85.70 92.80 42.80 57.10 50.00 71.40
22.96 40.13 67.61 40.83 20.29 55.82 80.02 68.70 40.33 49.96 60.77 67.11
vanillaeshop
17.80
28.50
21.40
48.15
54.70
35.70
34.38
2531
Table 5: E-commerce Websites Ranks and Mobile FriendlyTest E- Google store Page No Rank
Alexa Rank
Moz Rank
Moz Native GMF Domain Mobile Pass Auth. App
1
0
1,407,359
0.00
7.43
No
TRUE
2
1
220,757
4.29
13.23
No
TRUE
3
3
12,428
4.90
46.39
Yes
TRUE
4
0
101,521
4.79
33.06
Yes
FALSE
5
3
59,225
5.05
25.11
No
TRUE
6
7
176,734
5.38
36.91
No
FALSE
7
4
6,994
5.02
37.02
Yes
TRUE
8
0
7,310
6.04
40.90
Yes
TRUE
9
0
68,488
5.07
37.66
No
TRUE
10
3
27,424
5.98
39.17
Yes
TRUE
11
0
548,359
5.10
21.04
Yes
TRUE
12
3
520
5.52
59.00
Yes
TRUE
13
3
5,873,703
4.92
25.83
No
TRUE
Table 6: Websites Regional and Services Estore No
Payment
Delivery
Reginal Coverage
Aftersales Service
Website Language
1
2.5
8
7
4
5
9
2
6.5
8
10
10
10
9
3
8
8
7
4
10
9
4
1
2
7
4
10
9
5
4.5
8
7
4
10
10
6
4
8
7
4
10
9
7
5
8
7
10
10
9
8
6.5
8
7
10
10
9
9
5
10
7
10
10
9
10
8
8
9
2
10
10
11
9.5
8
7
6
10
10
12
10
8
7
4
10
9
13
8.5
8
8
4
5
9
Accepted Currency
performance indexes. When comparing the performance indexes for each E-commerce site, however, we find website performance is not always coincident with Twitter performance index (i.e., E-stores dokkanafkar.com and markavip.com). This implies the social big data can play a significant role affecting the performance assessment. The overall activity of Twitter accounts showed in Table 2 indicates that accounts activity can vary regardless of how many followers it has or how old the account is. The number of retweets actually reflects the account activity. In Table 1, the relatively new accounts (i.e., alhabib_shop, alwaneshop and matjarhk) that have a lower number of followers and tweets but were ranked as the top 3 for the retweet with significantly high percentages, see Table 8. The account alhabib_shop has a very small number of followers and tweets performed the best in retweet activity. It has only 76 tweets in the 28-day period but 10,229 retweets. Despite the low number of followers compared to top retweeting accounts and the very low tweeting rates, a huge amount of people retweeted their tweets. This indicates that those accounts’ followers are more active and are more loyal followers who often retweet (share) their tweets with their family, friends and followers. On the other hand, some accounts have a lower retweet rates like axiom_ksa and NamshiDotCom with 247 (nearly 9%) retweets from 2,751 tweets and 506 (about 24%) retweets from 2,082 tweets respectively. Even though some E-stores Twitter accounts have significantly high retweets, they may not perform well in other factors. Table 7: Website Performance Factosr Scores
In Table 7, we summarize the scores of three website performance indexes. These three indexes covered most of the factors that were previously identified by many studies and can influence the performance of an E-commerce site. Those commercial website evaluation tools, however, do not take social media into consideration. When comparing the result in Tables 4 and 7 (see the last column), we can compute the correlation coefficient 0.7765 between the averaged Twitter performance index and the averaged web
2532
E-store No.
Traffic Scores
Ranking and Mobile Friendly Scores
Regional and Services Scores
Average Website Performance
1
18.35
22.58
48.75
29.89
2
18.49
33.28
83.25
45.01
3
72.64
72.60
70.50
71.91
4
47.14
35.68
34.00
38.94
5
61.03
47.60
58.75
55.79
6
45.30
41.65
56.50
47.82
7
78.56
78.55
75.00
77.37
8
72.79
76.15
80.25
76.40
9
31.76
46.40
79.00
52.39
10
65.82
78.55
68.00
70.79
11
23.72
52.35
81.25
52.44
12
78.70
85.68
77.50
80.63
13
35.12
38.08
70.75
47.98
V.
Table 8: Number and Percentage of Retweets and Replies E-store No. 1
Retweets (%)
Replies (%)
10,229 (13,459.21%)
37 (48.68%)
2
16,961 (7,504.87%)
32 (14.16%)
3
77 (6.06%)
1,238 (97.40%)
4
626 (447.14%)
97 (69.29%)
5
17 (24.64%)
56 (81.16%)
6
1,229 (299.76%)
186 (45.37%)
7
1,777 (62.29%)
2,696 (94.50%)
8
19,481 (394.75%)
4,708 (95.40%)
9
247 (8.98%)
2,657 (96.58%)
10
661 (156.64%)
381 (90.28%)
11
40,826 (11,664.57%)
158 (45.14%)
12
506 (24.30%)
1,805 (86.70%)
13
559 (388.19%)
94 (65.28%)
Replying to, or mentioning, the followers is essential in the E-commerce environment to engage with the followers and answer their questions. This factor is also used to calculate the AE index. Table 8 shows the number and percentage of account’s replies. It indicates that the top period tweets E-Stores are also top E-stores in replies except for SouqKSA. Most of their tweets are replies in an average percentage of 95% meaning they are actively engaging with their followers. These E-stores also tend to score higher in website performance factor. However, SouqKSA outperformed them in website factor, which may explain why SouqKSA does not focus on replying to followers. Based on the results of our preliminary evaluation model, extrastores.com is the best performing E-commerce website by a score of (77.37+80.02)/2=78.7. According to Darasha’s online article, Extra invested in their E-commerce website, which started on 2011 (claimed to be the first in the country) and is growing by 300% annually [38]. The next best performing E-stores are also well-known E-stores in the region. Most likely, the E-stores with high Twitter performance scores have similar high website performance, which indicates that they work to improve their E-commerce efforts to remain in competition. The rest of medium or low performance score E-stores need to work on enhancing their efforts, especially in regional and services factors as they are performing poorly even with good scores on other factors. In addition, they need to find solutions to increase traffic to their websites as it plays a key role in the overall performance of E-commerce. Perhaps a good way is by increasing the Twitter WS factor that focuses on utilizing Twitter to bring visitors to the E-store’s website via links in the tweets. WS factor for some E-stores are extremely low and need to be improved. Therefore, its weight might be recalculated. Some E-stores have significantly high retweets, but they are not enough to perform well if not accompanied with high scores in other factors.
2533
CONCLUSIONS
In this paper, we have shown that social big data can be utilized for performance evaluation of E-commerce websites. Twitter data was collected for social data analytic, and several Twitter performance indexes were proposed. We also evaluated the E-commerce website performance using several commercial evaluation tools. Based on our study, the correlation coefficient is 0.7765 between the averaged Twitter performance index and the web performance index. In the future, we plan to include more E-commerce sites, various periods, more factors like link validation and page speed, and add other social media channels such as Facebook for our social big data analysis. Feature selection techniques will be used to determine which Twitter performance index is more effective or feature extraction techniques like principle component analysis to create new performance indexes. The ultimate goal would be to design and develop a new tool that leverages the social media data to dynamically evaluate the performance of E-commerce sites. During our study, we found that the website traffic information is not always accessible to public. Other useful information such as abandoned shopping cart and successful sales are not available to us either. A marketing strategy would be needed to convince the website owners allowing us to obtain those currently unreachable data. Finally, although our study was conducted using the Ecommerce sites in Saudi Arabia, the proposed framework can be adopted in other countries and regions with a different set of regional related factors or even different domains other than E-commerce. REFERENCES [1]
S. Akter and S. F. Wamba, “Big data analytics in Ecommerce: a systematic review and agenda for future research,” Electron. Mark., vol. 26, no. 2, pp. 173–194, 2016.
[2]
A. Al Matar and Q. Gao, “Social Media’s Impact on E Commerce : How to Adopt Social Media for E-Commerce Marketing,” 2011, pp. 1–6.
[3]
H. Alshehri and F. Meziane, “The Impact of Social Influence and Third Party Endorsement on Online Shopping In Saudi Arabia,” J. Internet E-bus. Stud., vol. 2015, pp. 1–9, 2015. M. N. Hajli, “A study of the impact of social media on consumers,” Int. J. Mark. Res., vol. 56, no. 3, pp. 387–404, 2013.
[4] [5]
[6]
[7] [8]
E. Makki and L.-C. Chang, “The Impact of Mobile Usage and Social Media on E-Commerce Acceptance and Implementation in Saudi Arabia.,” in The 2015 International Conference on e-Learning, e-Business, Enterprise Information Systems, and e-Government (EEE’15), 2015, pp. 25–30. R. J. Assaf, N. A. Noormohamed, and M. A. Saouli, “Rethinking Marketing Communication – Using Social Media to Attract College Consumers in the Middle East,” Compet. Forum, vol. 10, no. 2, pp. 112–117, 2012. M. A. M. I. Alafeef, “The Impact of Social Media on Sales Promotion in E-commerce Companies ( Case Study – KSAAl Baha City ),” vol. 6, no. 16, pp. 197–205, 2014. H. Liu, V. V. Krasnoproshin, and S. Zhang, “Combined Method for E-Commerce Website Evaluation Based on Fuzzy
Neural Network,” Appl. Mech. Mater., vol. 380–384, pp. 2135–2138, 2013. [9] [10]
[11] [12] [13] [14]
[15] [16]
[17]
[18] [19]
[20]
[24]
Y. Wang, H. Ding, and G. G. Xu, “E-Commerce Site Evaluation Based on Neural Networks,” Appl. Mech. Mater., vol. 263–266, pp. 2678–2681, 2012. M. B. Alotaibi, “E-Commerce Adoption in Saudi Arabia: an Assessment of International, Regional and Domestic Web Presence,” Int. J. Inf. Technol. Comput. Sci., vol. 5, no. 2, pp. 42–56, Jan. 2013. J. A. Rodríguez-Pineda, “WebQual: A Web Site Quality Instrument,” University of Georgia, 2000. A. Parasuraman, V. A. Zeithaml, and L. L. Berry, “Servqual: A Multiple-Item Scale For Measuring Consumer Perc,” J. Retail., vol. 64, no. 1, p. 12, 1988. H. S. Al-Khalifa, “A framework for evaluating university mobile websites,” Online Inf. Rev., vol. 38, no. 2, pp. 166– 185, 2014. E. Makki and L.-C. Chang, “Understanding the Effects of Social Media and Mobile Usage on E-Commerce: An Exploratory Study in Saudi Arabia,” Int. Manag. Rev., vol. 11, no. 2, pp. 98–109, 2015. B. J. Jansen and M. Zhang, “Twitter Power:Tweets as ElectronicWord of Mouth,” J. Am. Soc. Inf. Sci. Technol., vol. 60, no. 11, pp. 2169–2188, 2009.
[26]
[27] [28] [29]
N. Schoonderwoerd, “4 Ways How Twitter Can Keep Growing - PeerReach Blog,” 2013. [Online]. Available: http://blog.peerreach.com/2013/11/4-ways-how-twitter-cankeep-growing/. [Accessed: 20-Jan-2015]. R. AlGhamdi, S. Drew, and T. Alhussain, “A Conceptual Framework for the Promotion of Trusted Online Retailing Environment in Saudi Arabia,” Int. J. Bus. Manag., vol. 7, no. 5, pp. 140–150, Feb. 2012. S. a Al-Hudhaif and A. Alkubeyyer, “E-Commerce Adoption Factors in Saudi Arabia,” Int. J. Bus. Manag., vol. 6, no. 9, pp. 122–133, Sep. 2011. R. Alghamdi, A. Nguyen, J. Nguyen, S. Drew, and R. Al Ghamdi, “Factors Influencing Saudi Customers’ Decisions to Purchase from Online Retailers in Saudi Arabia: A Quantitative Analysis,” in IADIS International Conference eCommerce, 2011, no. Citc 2007, pp. 153–161. R. Alghamdi, A. Nguyen, and V. Jones, “A Study of Influential Factors in the Adoption and Diffusion of B2C ECommerce,” Int. J. Adv. Comput. Sci. Appl., vol. 4, no. 1, pp. 89–94, 2013.
[21]
S. S. Abed, Y. K. Dwivedi, and M. D. Williams, “SMEs’ Adoption of E-commerce Using Social Media in a Saudi Arabian Context : a Systematic Literature Review,” Int. J. Bus. Inf. Syst., vol. 19, no. 2, pp. 159–179, 2015.
[22]
M. R. Shihab, S. Wahyuni, and a. N. Hidayanto, “Analysis of trust presence within e-commerce websites: A study of Indonesian e-commerce websites,” in Proceedings - ICACSIS 2014: 2014 International Conference on Advanced Computer Science and Information Systems, 2014, no. October, pp. 133–138. D. Weatman, “Use Social Media to Manage Your Online Reputation and Support Your Brand,” Career Plan. Adult Dev. J., vol. Summer, pp. 107–114, 2016.
[23]
[25]
2534
A. Talpäu, G. Brätucu, and T. Viera, “Top Romanian Brands Online Presence - behavior and success strategies,” Rom. J. Mark., vol. apr-iun201, no. 2, pp. 13–23, 2013. A. O. Alotaibi and C. Bach, “Consumer Awareness and Potential Market for e-Commerce in Saudi Arabia: Challenges and Solutions,” in ASEE 2014 Zone I Conference, 2014, pp. 1–5. E. Makki and L.-C. Chang, “E-Commerce in Saudi Arabia: Acceptance and Implementation Difficulties,” in The 2014 International Conference on e-Learning, e-Business, Enterprise Information Systems, and e-Government (EEE’14), 2014, pp. 114–120. D. I. Zahran, H. A. Al-nuaim, M. J. Rutter, and D. Benyon, “a Comparative Approach To Web Evaluation and Website Evaluation,” Int. J. Public Inf. Syst., vol. 1, pp. 21–39, 2014. T. Guzzo, F. Ferri, and P. Grifoni, “ECA: An E-commerce Consumer Acceptance Model,” Int. Bus. Res., vol. 8, no. 1, pp. 145–156, 2014. S. Jamison-Powell, C. Linehan, L. Daley, A. Garbett, and S. Lawson, “‘I can’t get no sleep’: discussing #insomnia on Twitter,” in Conference on Human Factors in Computing Systems - Proceedings, 2012, pp. 1501–1510.
[30]
M. J. Culnan, P. J. McHugh, and J. I. Zubillaga, “How Large U.S. Companies Can Use Twitter and Other Social Media To Gain Business Value.,” MIS Q. Exec., vol. 9, no. 4, pp. 243– 259, 2010.
[31]
M. E. Hambrick, J. M. Simmons, G. P. Greenhalgh, and T. C. Greenwell, “Understanding Professional Athletes’ Use of Twitter: A Content Analysis of Athlete Tweets.,” Int. J. Sport Commun., vol. 3, no. 4, pp. 454–471, 2010.
[32]
D. M. Romero, B. Meeder, and J. Kleinberg, “Differences in the mechanics of information diffusion across topics: idioms, political hashtags, and complex contagion on twitter,” in Proceedings of The 20th International Conference on World Wide Web, 2011, pp. 695–704.
[33]
J. Yang and S. Counts, “Predicting the Speed , Scale , and Range of Information Diffusion in Twitter,” in Fourth International AAAI Conference on Weblogs and Social Media, 2010, pp. 355–358. T. Sakaki, M. Okazaki, and Y. Matsuo, “Earthquake shakes Twitter users: real-time event detection by social sensors,” in Proceedings of The 19th International Conference on World Wide Web, 2010, pp. 851–860. D. Davidov, O. Tsur, and A. Rappoport, “Enhanced Sentiment Learning Using Twitter Hashtags and Smileys,” in Proceedings of the 23rd International Conference on Computational Linguistics: Posters, 2010, no. August, pp. 241–249.
[34]
[35]
[36] [37]
[38]
A. Bruns and S. Stieglitz, “Towards more systematic Twitter analysis : metrics for tweeting activities.,” Int. J. Soc. Res. Methodol., vol. 16, no. 2, pp. 91–108, 2013. E. Makki and L.-C. Chang, “E-Commerce Acceptance And Implementation In Saudi Arabia: Previous, Current And Future Factors,” Int. J. Manag. Res. Bus. Strateg., vol. 4, no. 3, pp. 29–44, 2015. Brinda Darasha, “Saudi retailers could miss e-commerce bandwagon,” 2016. [Online]. Available: http://www.argaam.com/en/article/articledetail/id/431607. [Accessed: 25-Aug-2016].
