A user experience evaluation of Amazon Kindle mobile application Azham Hussain, Emmanuel O. C. Mkpojiogu, Ja’afaru Musa, and Salah Mortada
Citation: AIP Conference Proceedings 1891, 020060 (2017); doi: 10.1063/1.5005393 View online: http://dx.doi.org/10.1063/1.5005393 View Table of Contents: http://aip.scitation.org/toc/apc/1891/1 Published by the American Institute of Physics
A User Experience Evaluation of Amazon Kindle Mobile Application Azham Hussain1, a), Emmanuel O.C. Mkpojiogu1, b), Ja’afaru Musa1, c) and Salah Mortada1, d) 1
Human-Centered Computing Research Lab, School of Computing, Universiti Utara Malaysia, 06010 Sintok, Kedah, Malaysia a) Corresponding author:
[email protected] b)
[email protected] c)
[email protected] d)
[email protected]
Abstract. There is a dramatic increase in the development of mobile applications in recent years. This makes the usability evaluation of these mobile applications an important aspect in the advancement and application of technology. In this paper, a laboratory-based usability evaluation was carried out on the Amazon Kindle app using 15 users who performed 5 tasks on the Kindle e-book mobile app. A post-test questionnaire was administered to elicit users’ perception on the usability of the application. The results demonstrate that almost all the participants were satisfied with services provided by the Amazon Kindle e-book mobile app. On all the four user experience factors examined, namely, perceived ease-of-use, perceived visibility, perceived enjoyabilty, and perceived efficiency, the evaluation outcome shows that the participants had a good and rich mobile experience with the application.
INTRODUCTION The rapid and growing amount of mobile apps in the Google Play and Apple stores have dared and prompted programmers to develop applications of maximum quality in order to fit into the rising competitive apps markets. There are a lot of measurable aspects on the quality of mobile apps, and the notably significant one is usability [78][11]. The development of these mobile apps should take into consideration several design constraints, for example, limited and restricted resources, the Web connectivity problems, data entry models, smallness of screen size and a host of other diverse showings of mobile devices [1].However, usability of mobile device app has become an important issue because several software products that previously ran using PCs (laptops and desktops), are currently run using smart phone technologies. From the perspective of users, usability is essentially an evaluation of the effectiveness and efficiency of a system. Usability might be categorized in accordance with the definition of ISO 9241-11 under 3 central factors, and they are: user satisfaction, effectiveness, and efficiency of a product in a specific context of use [2][5][12]. Similarly, usability testing is also very important in measuring the efficiency, satisfaction and effectiveness of the interface of a mobile device as users interact with the mobile device in actual usage conditions [3][15]. The nonevaluation of the usability of mobile apps during the development and design phases of such apps frequently causes users' disappointment, frustration, dissatisfaction, difficulty in use of the app, much expending of users’ valuable resources, denial of use, and outright rejection of the products. Practical usability issues normally crop up in actual usage of the application. This makes usability an aspect of software quality. There is a strong demand for it to be incorporated during the design and development of all software products [18]. Amazon.com designs and markets the Amazon Kindle which features a series of e-readers. Amazon Kindle devices have the capacity to help clients browse, download, buy, and go through e-books, newspapers, magazines, and other digital media with the medium of wireless networking for kindling the store. Amazon subsidiary, Lab126 develops the hardware platform. The hardware platform started working as a single device, and now it has a range of devices that includes e-readers equipped with E-Ink electronic paper displays as well as Android-based tablets along with color LCD screens. All Kindle devices are incorporated with the Kindle Store to have the content. Also, the Kindle Store contains circulation, which encompasses four million e-books in the USA for instance. This present study focuses on the usability evaluation of e-book reader. An e-book can be an electronic digital edition of printed publications. An e-book could be read utilizing e-book reader (e-reader), that is, a digital system or app that establishes electronic text to the user. E-books may also be read on an individual computer (PC) and mobile phones. This study is concentrating on the Amazon Kindle e-book software for mobile phones (not the app for PCs) [14]. The objective
The 2nd International Conference on Applied Science and Technology 2017 (ICAST’17) AIP Conf. Proc. 1891, 020060-1–020060-7; https://doi.org/10.1063/1.5005393 Published by AIP Publishing. 978-0-7354-1573-7/$30.00
020060-1
of this study is to evaluate the usability of Amazon Kindle mobile application. The remaining part of this paper is organized as follows: part II is on the related works, part III is on the methods, part IV is on results and discussion, while the last part, part V is on the conclusion.
RELATED WORKS In today’s world, mobile devices usage have swiftly improved and increased immensely within the last few years. These devices have grown to be more advanced and guides users to attempt and achieve wide-ranging tasks. Mobile phones and their distinct applications provide spacious benefits to their operators in terms of portability, position consciousness (location awareness) and also accessibility. Reasonably priced items and their improvements inside the computer hardware along with software program capabilities involving smart phones in the market especially the handhelds have led to the enormous extension of mobile phone in the associated marketplaces. This has generated massive amounts of mobile applications that have been developed and deployed within the last few years [4]. The essence of using the mobile device is not limited to only communication, but it has become a fundamental tool in our day-to-day activities. Absolutely, it can be said that the possibility of the future network lies in mobile terminal. Presently, Android operating system is gaining popularity, especially in the context of smart phones. A lot of free access to open sources and development tools are available. This result in the development of many useful and affordable applications that influences a lot of people to use the Android operated smart phones. Furthermore, it gives better and suitable hardware platform for developers, and throughout the process, developers are able to apply lesser mechanisms to turn their ideas into reality, and further development can be achieved through the Android operating system [6]. Similarly, users of mobile phones have the opportunity to enjoy movies or play shared games featured in their mobile phone. These innovative characteristics regarding mobile applications help users to perform different varieties of activities using their mobile devices. The success of mobile applications depends on achieving a high level of customers’ satisfactions; as a result, usability evaluation is a compulsory process in making sure that mobile application has the features of acceptable practicality, effectiveness, ease of use, efficiency and usefulness, particularly from the perspective of users [9]. Equally, a few publishers blend articles with elements of multimedia, and these elements encompass web pictures, videos, links, menus and audio in the creation of interactive e-books. The Nielsen Norman Group carried out a usability evaluation for websites and mobile apps that were particularly designed for the IPad tablet. Users who were able to go from one page to another that contains a summation of an article by sipping the page or by dragging the surface, can read the whole article [10]. Likewise, mobile applications which operate on mobile devices have faster growth and pervasive information access anywhere and at anytime. For instance, many mobile applications access Internet services via mobile devices. In the business world, some applications are related to m-commerce, such as the advertising and mobile banking applications. These applications improve business electronic items. Clients are able to check their bank accounts balance and can perform other business transactions with their mobile phones [12-13][21]. In addition, the use of a digital device is quite old. Alan Kay pioneered prototypes for e-reading around the year 1960 and a few generations of devices were embodied later on. Examples are the Rocket e-book, Apple Newton, and the Amazon Kindle. More so, there is some advancement, which took place in device technology, such as: displays, batteries, and CPU among others. Besides, publishers are able to offer several digitally published solutions, which tend production towards publishing, and the examples include AppCross and others [14]. Novices have the option of choosing the cheapest one and ignoring the characteristics of the application of readers. Even though the topics included in the book is similar, the user interface of the e-textbook application are different. Learning the ways of making a note, searching for a keyword, highlighting, and navigating the book in every application is mandatory for users. Suggestions have been made to standardize e-textbook design guidelines [16][17], which are not observed as mandatory; as a result, the interfaces have considerable differences. Wilson et al. [17] put forward the guidelines, which have the basics on the assessments regarding e-books on webpage as well as for outdated readers of e-book [17]. In addition, manufacturers of mobile device intend to implement their own usability boundaries. An instance is mentioned here; the Guidelines of Apple iOS Human Interface mentions that the platform of iOS features need to be taken into account during the process of application development, and should show different resolutions and dimensions, collaboration with multi-touch screen, device orientation gestures and changes, such as pinch, flick and tap. Furthermore, Apples reviews applications that are presented for the App Store, and the basis of their acceptance depends on conformance with the characteristics outlined in the guideline. Similarly, Google developed the Android user interface guidelines. These guidelines guide developers to ensure features are properly
020060-2
placed, and to guide the dimension and placement of icons and buttons, touch signals, menus in contexts and their responsiveness, size, simplicity, and formatting of text, and particular pieces of messages. The said guidelines also mention the ways these characteristics need to be taken into account at the time of the design, development, growth and testing of Android applications [1][19]. Moreover, usability evaluation is a part of a bigger effort that targets at improving the profitability, designing decisions and the minimizing of the disappointment and mistakes for users [20]. In order to achieve the goal of evaluation, systems require building a suitable design. A number of formative usability evaluations need to be done for all products severally before the product gets released. Even after it has been released, several summative usability evaluations should still be performed as well. This is geared towards ultimately eliminating the frustrations and problems related to design from the perspective of the users and the usability issues that will follow in actual usage of the apps. The result is that customers gets valuable, effective, efficient and pleasing product in return [16][22]. Numerous studies have put emphasis on the usability of mobile devices. Gafni [23] and Ryu [24] express the opinion that difficulties encountered are due to the physical constraints regarding wireless networks and mobile devices, including the time of conducting and the designing of usability testing studies for mobile applications [13]. The study of Zhang et al. [13] demonstrates that mobile usability is faced with several mobile-related challenges like: mobile context, connectivity, small screen size, different display resolutions, limited processing capability and power, and data entry methods.
METHOD The usability evaluation was conducted at a quiet and controlled place within the UUM library and at the DPP Proton Students’ Residential Hall gallery and cafeteria. The users were allowed the freedom to perform the tasks while sitting and/or standing. There were two moderators; one was in-charge of giving the study tasks orally to the users while the other was in-charge of capturing the test sessions with a mobile phone camera. After the task had been communicated by the moderator to the participants, no constraints was imposed on the user during the test session so as to allow for a free and conducive environment for the study participants.
Tools The participants used the Android phones that were provided by the moderator to perform the tasks without further adjustment of settings. Each test session was recorded on video. The camera records the smart phone’s screen as well as the respondent’s movements throughout the session. In addition, a stopwatch was used to record the time taken for each task.
Task and Test Description Fifteen (15) participants were used in the test session. All participants were students. A touch screen was used to carry out 5 tasks. The moderator recorded the entire sessions on video while the participants were engaged with the tasks. The usability test was conducted individually, that is, one user at a time. After completing the tasks, the participants completed a post-test questionnaire. The following are the five tasks involved in the test: i) Search Text, ii) Bookmarking, iii) Make a Note, iv) Sharing e-books, and v) Change Text Size. Prior to the start of the test, the moderator encouraged participants to use the “think aloud” protocol while they are in the process of executing the tasks. The participants were also instructed to move between tasks and do whatever they would naturally do. This was done in order to capture users’ behavior at the time of navigating from one page to another. Users’ reactions to the Kindle application were captured with the use of video camera. Conducting the usability test involved the following: Execution of the 5 tasks by the 15 participations who evaluated the Kindle application. These participants were randomly chosen to test the application. The moderator records all test sessions on video camera, capturing the errors that occurred and time taken. The measurement involves counting the number of errors the participants made when attempting to complete a task. The operation time is the time taken to complete a task. The time to complete each task is recorded with a stopwatch. Task success (whether or not a participant successfully completes a task) is also recorded. Participants are given up to one minute to complete each task (pilot testing is done to determine that 1 minute is sufficient for task completion). When the 5 tasks are completed, the participants were immediately given a post-test questionnaire to respond to so as to measure each factor of interest in the usability test. Finally, quantitative data gathered on the operation (task) time, error frequency, and
020060-3
task success rate were computed. The results of this test were analyzed using SPSS version 23. Descriptive statistics such as mean and frequency distribution were used to analyse the collected data and present results [26]. This present paper however reports on only the results of the perceived usability captured using the subjective post-test questionnaire.
RESULTS The participants’ demographic information is as follows: of the 15 participants involved in the study, 10 were male and 5 were female. Nine of the participants had their ages ranged from 18 – 29; five were within the age range of 30 – 39, while one was aged within the 40 – 49 age brackets. Categorizing the participants based on their past experience on the use of the application, 9 participants were novice users, 5 were moderate users, while only 1 was an experienced user. One of the participants was an undergraduate; 6 were master’s students, while 8 were PhD candidates. Perceived Ease of Use: Table 1 shows the descriptive statistics for the perceived ease of use of the application. As can be seen, majority of the participants agreed that the application was easy for them to use. The mean and standard deviation per item is as shown in Table 1. The use of the application was not difficult and challenging to most of the users. Interestingly, usability is centered on the ease of use of an application. If an application is difficult to use, users generally will abandon such application in frustration and go for an easier alternative. This is why users are disloyal to many products in the market today.
Item EU1 EU2 EU3 EU4 EU5 EU6 EU7
Item
N
Mean
15 15 15 15 15 15 15
4.33 4.33 4.13 4.26 2.60 4.20 4.13
N
Mean
V1 V2 V3 V4
15 15 15 15
4.20 4.40 4.20 4.26
Item
N
Mean
EA1 EA2 EA3 EA4
15 15 15 15
4.26 4.40 4.26 4.33
TABLE 1. Perceived Ease of Use Min Max Likert Scale S. Disagree Neutral Disagree .816 2 5 0 1 0 . 816 2 5 0 1 0 .915 2 5 0 1 2 .883 2 5 0 1 1 1.121 1 5 2 6 4 .861 2 5 0 1 1 .990 2 5 0 2 0
Standard Deviation
Standard Deviation .861 .828 .774 .798
TABLE 2. Perceived Visibility Min Max Likert Scale S. Disagree Neutral Disagree 2 5 0 1 1 2 5 0 1 0 2 5 0 1 0 2 5 0 1 0
TABLE 3. Perceived Enjoyability Standard Min Max Likert Scale Deviation S. Disagree Neutral Disagree .961 2 5 0 1 2 1.182 1 5 1 0 2 1.032 1 5 1 0 0 1.112 1 5 1 0 1
Agree 7 7 6 2 2 7 7
Agree 7 6 9 8
Agree 4 1 7 4
S. Agree 7 7 6 1 1 6 6
S. Agree 6 8 5 6
S. Agree 8 11 7 9
Perceived Visibility: The perceived visibility result is illustrated in Table 2. The table shows that majority of the study participants agreed that the application’s interface was visible to them. The mean and standard deviation per item is as shown in the table. The result implies that the features and functionalities on the interface were clearly seen
020060-4
by users and that they can access these features without difficulties. Visibility is very important as it enhances an app’s effectiveness and efficiency. An interface that is blurred or unclear frustrates users, wastes their resources (time and mental efforts) and affects them achieving their goals. It leaves users with dissatisfaction that may eventually lead to their disloyalty to the application. Perceived Enjoyability: The descriptive statistics for the perceived enjoyability of the application is shown in Table 3. The result shows that most of the participants enjoyed using Kindle mobile application. Majority of them agreed that they enjoyed the Kindle e-book. The mean and standard deviation per item is as shown in the table. Perceived enjoyment improved user satisfaction and promotes their loyalty to an application. It makes users to be positively disposed and attracted to an application and to have a feeling of fun and comfort while using the product. Satisfaction makes users to have a positive attitude towards a product as well as a positive word-of-mouth that enhances product loyalty. Perceived Efficiency: The perceived efficiency of the mobile application is shown in Table 4. Most of the participants agreed that the application’s interface was efficient and that they expended minimal efforts using the application. The mean and standard deviation is as shown in the table. Efficiency is very crucial to an application. If users spend too much time on an application before achieving their goals, such users will dislike the application. The more an application is efficiency the more satisfying it will be to the users. The likeability rating of an application is a product of the users’ perception of its efficiency. Does the application demand so much mental efforts? How much time do users expend on the application in carrying out their tasks? Does the user leave the application fulfilled or frustrated? These questions are very important to the user as he is interested in achieving his goals within a limited timeframe and minimal mental resources. Efficiency affects user satisfaction towards an application and their loyalty to the application. TABLE 4. Perceived Efficiency (PE) Item
N
Mean
Standard Deviation
Min
Max S. Disagree
E1 E2 E3
15 15 15
4.06 4.40 4.46
.798 .828 .833
2 2 2
5 5 5
0 0 0
Likert Scale Disagree Neutral
1 1 1
1 0 0
Agree
S. Agree
9 6 5
4 8 9
CONCLUSION This study evaluated the Amazon Kindle mobile application on four user experience factors, namely, perceived ease of use, perceived visibility, perceived enjoyability, and perceived efficiency. The result shows that the application provided an enriching user experience to most of the users who participated in the study. Most of the users perceived that the mobile application satisfied their needs in terms of the four usability qualities evaluated in the study. However, since some of the participants (though in the minority) had issues with the application, it suggests that the application still need to be further and continually improved upon to accommodate the challenges of this class of users who are certainly in large number when their sample is projected to the population. There is no end to the enhancement of a product.
020060-5
REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.
F. Nayebi, J.-M. Desharnais and A. Abran, (2012),”The state of the art of mobile application usability evaluation”. 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), (2012), pp. 1– 4. A. Hussain, E.O.C. Mkpojiogu and Z. Hussain, “Usability evaluation of a web-based health awareness portal on Smartphone devices using ISO 9241-11 model”, Jurnal Teknologi (Sciences & Engineering), 77 (4), 1-5 (2015). J. Williams, “Developing performance support for computer systems: a strategy for maximizing usability and learnability”, (2004). R. Harrison, D. Flood and D. Duce, “Usability of mobile applications: literature review and rationale for a new usability model”, 1–16 (2013). A. Hussain and E.O.C. Mkpojiogu, “The effect of responsive web design on the user experience with laptop and smartphone devices”, Jurnal Teknologi (Sciences & Engineering), 77 (4), 41-47 (2015). C. Wang and C. Huang, “A study of usability principles and interface design for mobile e-books. Ergonomics, 58 (8), 1253–1265 (2015). A. Hussain, E.O.C. Mkpojiogu and F. Hassan, “Systematic review of mobile learning applications for children”, Proceedings of the 2nd International Conference on Information and Communication Technology for Transformation (IC-ICT4T’16), Kota Kinabalu, Sabah, Malaysia (5-7 April, 2016). E.O.C. Mkpojiogu, N.L. Hashim and R. Adamu, “Observed demographic differentials in user perceived satisfaction on the usability of mobile banking applications”, Proceedings of the 8th Knowledge Management International Conference (KMICe’16), Chiang Mai, Thailand, (29-30 August 2016), pp. 263-268. A. Behler and B. Lush, “Are you ready for e-readers?” The Reference Librarian, 52 (1-2), 75–87 (2010). R. Budiu and J. Nielsen, “Ipad app and website usability: research findings a year after launch”. Nielsen Norman Group, 1–116 (2011). A. Hussain, E.O.C. Mkpojiogu and F. Hassan, Usability dimensions and sub-dimensions for the evaluation of m-learning apps for children: A systematic review. Jurnal Teknologi (Sciences & Engineering). A. Hussain and E.O.C. Mkpojiogu, “Usability evaluation techniques in mobile commerce applications: A systematic review. Proceedings of the International Conference on Applied Science and Technology (ICAST’16), Kedah, Malaysia. AIP Conf. Proc. 1761 (1), (2016), pp. 020049. D. Zhang and B. Adipat,” Challenges, methodologies, and issues in the usability testing of mobile applications”. International Journal of Human-Computer Interaction, 18 (3), 293–308 (2005). G. Golovchinsky, “Reading in the office”. Research Advances in Large Digital Book Repositories, 21–24 (2008). A. Hussain and E.O.C. Mkpojiogu, “An application of ISO/IEC 25010 standard in the quality-in-use assessment of an online health awareness system”, Jurnal Teknologi (Sciences & Engineering), 77 (5), 9-13 (2015). N. Ahmad, M. Boota and A. Masoom, “Smart phone application evaluation with usability testing approach”. Journal of Software Engineering, 7 (12), 1045 (2014). R. Wilson, M. Landoni and F. Gibb, “A user-centered approach to e-book design”. The Electronic Library, 20 (4), 322–330 (2002). E.O.C. Mkpojiogu and N.L. Hashim, “The impact of users’ age, gender, education and experience on their satisfaction perception of m-banking apps’ usability”, ARPN Journal of Engineering and Applied Sciences. H. Hoehle and V. Venkatesh, “Mobile application usability: conceptualization and instrument development”. MIS Quarterly, 39 (2), 435–472 (2015). J. Rubin and D. Chisnell, “Developing the Test Plan”. The Handbook of Usability Testing, (2008).
020060-6
21. A. Hussain, E.O.C. Mkpojiogu and F.M. Kamal, “A systematic review on usability evaluation methods in mcommerce apps”, Journal of Telecommunication, Electronic & Computer Engineering (JTEC), 8 (10), 29-34 (2016). 22. A. Zaman and M. Bhuiyan, “Usability evaluation of the MumIES (Multimodal Interface based Education and Support) system for the children with special needs in Bangladesh”. In 2014 International Conference on Informatics, Electronics & Vision (ICIEV), (IEEE, 2014), pp. 1-4. 23. R. Gafni,”Usability issues in mobile-wireless information systems”. Issues in Informing Science and Information, 6, 755–769 (2009). 24. Y. Ryu, “Development of usability questionnaires for electronic mobile products and decision making methods”, (2005). 25. A. Hussain, E.O.C. Mkpojiogu and F.M. Kamal, “Mobile video streaming applications: A systematic review of test metrics in usability evaluation”, Journal of Telecommunication, Electronic & Computer Engineering (JTEC), 8 (10), 35-39 (2016). 26. K.R. Mwaura, “Technology Acceptance of Cloud Computing in ICT Departments of the Kenya Government Ministries”. (University of Nairobi, 2013).
020060-7
