use style: paper title

User Experience of SIPEBI as Indonesia Cultural Edutainment web portal Santo Tjhin Faculty of Design Visual Universitas Multimedia Nusantara Tangerang, Indonesia [email protected]

Dony Saputra School of Business and Management, School of Design Bina Nusantara University Jakarta, Indonesia [email protected]

Abstract— this is the third phase of research on creating Iconic Pusat Edukasi Budaya Indonesia (SIPEBI) as Indonesia cultural edutainment web portal. On this research we emphasized more to User Experience (uX) measurement based on Jeff Sauro usability metric, in this research we only use completion rate, usability problems, errors, page view/click and single usability metrics of SIPEBI. The method that we use is an explanatory with an experimental approach by measuring user experience based on usability metrics as integrated part of SIPEBI creation as an Indonesia educational web portal. Result of this research is based on single usability metrics in user experience overview it supposed to be redesign as the error, completion rate and satisfaction doesn’t comply with standard of usability metrics. Keywords—User Experience, edutainment, web portal.

I.

SIPEBI,

usability

metrics,

INTRODUCTION

In the process of teaching and learning for Indonesian national plus or international schools, teachers are emphasized to give alternative way of teaching and assessments in addition to the main teaching and assessments on students’ learning process. Generally, type of teaching and learning process use one way communication where teacher explain and student pay attention to the teacher. By the development of ICT, especially internet, it change into two way learning process where student explore more of their learning material, with the implementation of web portal using edutainment concept. This type of portal provides games based on students’ lessons as well as quizzes, tests until the results of what they have learn in a form of passing certificate. Web Portal like www.bcc.co.uk/bitesize/ks2, gives lessons on English, science and mathematics, while www.ixl.com provides alternative lessons for mathematics and English. Whereas for Social Sciences related to local and Indonesian culture, only provided in a portal by Indonesian Ministry of education that

is in http://belajar.kemdiknas.go.id/ provided, containing resources on Indonesian cultures map but not in the form of edutainment content based on interactive games or material. Edutainment concepts are learning concepts that is entertaining and fun. In edutainment, students learning process is not only by reading and listening teachers’ lectures but they also try or experimented with multimedia content as well as interactive games without losing the lessons’ contents. Besides, teachers and schools at primary level are emphasized to design and give lessons aiming on increasing the interests and interactivity of students. The conventional teaching and learning process can be implemented with teaching and learning concept of playing in the form of edutainment. From the edutainment concept, students can explore lessons that given by teachers and learn more using the technology in self-directed learning concept. As an alternative assessment, the result of those lessons in the web portal can be printed out by the teacher as authentic students’ work portfolio alongside with student’s report card distribution.

A. Research Question So, based on the above introduction, the problems that will be used as the reference for problem Solving are as follow: 1. What is user experience on SIPEBI web portal? 2. How do we measure user experience on SIPEBI web portal?

B. Research Limitation The coverage of this research will only added interactivity design of SIPEBI web portal , containing third phase development of edutainment application for 2 Indonesian cultures that is Betawi and Aceh. Sub menu of the web portal will be history, culture and national heroes. Every content consists of narration, video, quiz and game. The target of this research’s result is children ranging from the age of 6 to 9 or primary 1 to 3. II.

•

LITERATURE REVIEW

Edutainment, according to Jiri N and Josef T [4] is a special form of entertainment that allows the participants to be educated like getting new information from various fields in our life. Edutainment consist of 2 syllables which are education and entertainment, when combined it will be education that gives entertainment. Based on the above opinions, edutainment can be defined as a learning process that’s designed to fuse education with entertainment so that learning activities will be fun. There are 3 assumption concepts that will be the foundation for edutainment concept. The first assumption declares that positive feelings can accelerate learning process while negative feelings slow down and obstruct learning process. The second assumption declares that if someone can use his sense of thinking potential and emotions accurately he’ll make a leap in academic achievements that no one can think of. The third assumption declares that if every learners can be motivated accurately and are taught using the right way, in the way of respecting their way of learning, they all will achieve optimized learning result. Application of those 3 assumptions in edutainment able to give a learning system that is designed interactively, open and fun, not scaring and two way learning environment. This concept places the students as the center of learning process that involves relationship between the student, teacher, learning process and learning environment throughout SIPEBI web portal. User experience is the experience the product creates for the people who use it in the real world. User experience is about how product or service work on the outside when a person comes into contact with it, it is not inner working of product or services [2]. Usability is summed up as being a general quality of the appropriateness to a purpose of any particular artefact [1]. Usability metrics is a metric for usability evaluation that measure impact of good and bad usability to assess the quality of user experience [11]. Usability metrics that is use for usability evaluation are: • Completion Rates: Often called the fundamental usability metric, or the gateway metric, completion rates are a simple measure of usability. It's typically recorded as binary metric (1=Task Success and 0= Task failure). • Usability Problems (UI Problems) encountered (with or without severity ratings): Describe the problem and note

•

•

•

•

•

•

•

both how many and which users encountered it. Knowing the probability a user will encounter a problem at each phase of development can become a key metric for measuring usability activity impact and ROI. Knowing which user encountered it allows us to better predict sample sizes, problem discovery rates and what problems are found by only a single user. Task Time: Total task duration is the de facto measure of efficiency and productivity. Record how long it takes a user to complete a task in seconds and or minutes. Start task times when users finish reading task scenarios and end the time when users have finished all actions. Task Level Satisfaction: After users attempt a task, have they answer a few or just a single question about how difficult the task was. Task level satisfaction metrics will immediately flag a difficult task, especially when compared to a database of other tasks. Test Level Satisfaction: At the conclusion of the usability test, have participants answer a few questions about their impression of the overall ease of use. For general software, hardware and mobile devices consider the System Usability Scale (SUS), for websites use the SUPR-Q. Errors: Record any unintended action, slip, mistake or omission a user makes while attempting a task. Record each instance of an error along with a description. We can later add severity ratings to errors or classify them into categories. Errors provide excellent diagnostic information and, if possible, should be mapped to UI problems. Expectation: Users have an expectation about how difficult a task should be based on subtle cues in the task-scenario. Asking users how difficult they expect a task to be and comparing it to actual task difficulty ratings (from the same or different users) can be useful in diagnosing problem areas. Page Views/Clicks: For websites and web-applications, these fundamental tracking metrics might be the only thing you have access to without conducting your own studies. Clicks have been shown to correlate highly with time-on-task which is probably a better measure of efficiency. The first click can be highly indicative of a task success or failure. Conversion: Measuring whether users can sign-up or purchase a product is a measure of effectiveness. Conversion rates are a special kind of completion rate and are the essential metric in e-Commerce. Conversion rates are also binary measures (1=converted, 0=not converted) and can be captured at all phases of the sales process from landing page, registration, checkout and purchase. It is often the combination of usability problems, errors and time that lead to lower conversion rates in shopping carts. Single Usability Metric (SUM): There are times when it

is easier to describe the usability of a system or task by combining metrics into a single score, for example, when comparing competing products or reporting on corporate dashboards. SUM is a standardized average of measures of effectiveness, efficiency of satisfaction and is typically composed of 3 metrics: completion rates, tasklevel satisfaction and task time According to Goni [3], web portal is web site that were used as bridge that provides wide variety of resources, service, task and link to other website. Based on his research there will be two kinds of web portal that is horizontal portals and vertical portals. Whereas Horizontal portals is a type of portal that offered by internet service provider or search engine usually with general purposes such as yahoo or google. Whereas vertical portal is used for specific interest or field such as those portal for medical information. TABLE I. No. Title and author 1 Willyanto[10] Perancangan Aplikasi EEducation Berbasis Web Dengan Menggunakan Bahasa PHP 2 Ifan Kurniawan[2] Pemanfaatan EEducation untuk meningkatkan Kualitas Belajar

3

Oka A.S., Widya C and Ilman Z[7] Perancangan sistem edutainment berbasis web di sekolah menengah atas negeri 1 curup kabupaten rejang lebong

4

Sidiq Wahidi[8] Pembelajaran bahasa Arab berbasis edutainment (model ustadz Mohamad Nuaim pada kelas X di MAN Wonokromo

LITERATURE REVIEW Result E-Education System must be supported by the resource with qualities so that learning teaching process will become effective. They use web based application using PHP Programming Language utilizing e-education as a transport in learning in the field of computer is used in order to help students in finishing problems especially in the field of computer and lecture notes can be downloaded by students freely so that it can increase knowledge and insight of the students A design of an edutainment system for interactive learning that creates student’s interest to study. This system combines the element of education with entertainment to erase the feeling of boredom in the students towards the past learning way. This system is designed basing on web so it will ease up the user and utilize system networks available at SMA Negeri 1 Curup. Capturing the edutainment concept practiced by learning arabian language ustadz Mohamad Nuaim version that covers happiness of taught participants, way of thinking, their personal experiences, ability of

Adoption Reference on building eeducation application with PHP language.

the concept and reference how eeducation can increase learning qualities

Reference to make an edutainment system based on web with PHP and MySQL.

Reference on edutainment concept emphasizing in fun student’s learning process.

Pleret Bantul)

5.

6.

Zufri. T, Saputra. D, Tjhin. Santo [15] “Desain Portal Edutainment Budaya Indonesia sebagai sarana self-directed learning siswa di tingkat sekolah dasar”. Saputra. D, Tjhin. Santo , Zufri. T [8] “SIPEBI: An overview of cultural edutainment in web portal as a medium for students’ selfdirected learning and alternative’s assessment ”.

teachers in teaching, learning atmosphere, creative media, edutainment in learning and evaluation model Phase 1 of creating cultural edutainment web portal without implementation on quiz, game or certificate. Software using Flash, PHP.

Phase 2 of creating cultural edutainment web portal added implementation on quiz, game or certificate. Software using Flash, PHP.

Design and creation of cultural edutainment web portal.

Second phase design and creation of cultural edutainment web portal.

While in this third phase we add more on interactive design by giving a usability testing on internal test. III.

RESEARCH METHODOLOGY

The methodologies used in this research are experimental methods that will measure usability user experience on SIPEBI web portal. The research methods are divided into 7 stages: Deploy Web portal, completion rate test, Usability problem test , errors, page view/click, single usabaility metrics and conclusion. Figure 1 shows these stages:

Fig.1 Research Methodology

IV.

B. Design Portal Design for the portal will be shown using unified modeling language (UML) use case and activity diagram for menu and action inside the portal as shown in Figure 3 below.

DISCUSSION

This is results of usability test on user experience that done internally by researcher in using SIPEBI. It will be divide into divided into interactive design, Design Portal, usability measurement and conclusion.

A. Interactive design Base on Dr. Mark Ingham [6], Interactive design are divided into 13 components. In this research we use 3 component that is applied to this research. The first is feel experience that based on layout and components inside the portal that give feeling of interactivity to user that is Touch screen, button and switch. The second is define interactions that what area of human feel that touch by interactivity that is touch, hear and see. The third is positive emotions is the goal of this research and interactivity consist of happy, joy and fun. Interactivity design are shown by figure 2. Below.

Fig. 3 SIPEBI use case and activity diagram

Fig. 2 Interactivity Design

Fig. 3 explain the stages of using the portal. Students as users will open the homepage from iconic web portal. The homepage provides a button in the form of icon animation that contains Betawi culture, Aceh culture, Makassar/Bugis culture, Dayak culture and Flores culture. After clicking one of the icons, an animation of the origin of the selected culture will appear and another history, culture and national hero buttons from the selected culture will also appear. If the history button is clicked, an animation story that narrates the history will appear and narration, video, games and quiz menu will be flashed. If the quiz menu is selected, a series of questions that will be answered by the students will appear and by the end of the last question, the total score will be displayed and there is an option to print the score. If the culture is selected, introductory narration animation will appear and another set of music art, dance art, traditional clothes, houses, foods and drinks buttons will appear. After the buttons are chosen, a similar menu as the history button will be displayed.

Fig. 4 SIPEBI Display

Every picture in this page, as in Fig 2, is a moving animation that will display names from each tribe when magnified. The next menu will be displayed once clicked. When the page appears for the first time like shown in fig. 2 there would be an animation of betawi village appearances that was covered by metropolitan city, then there is the traditional house of betawi. After that there is the character si pitung that was running from his house and sees the background with narration and background music. After that, 3 arranged and animated buttons appeared that is history, culture and national heroes. The third picture in fig.2 is a print out of certificate as an alternate assessment after they’ve done the quiz or game, It will included passing score and level of quiz or game they have passed. C. Usability Testing on User Experience • Usability problem test In this test we specified design and layout user interface problem as button, link, option and check box button with black box testing as follow: TABLE II. No 1

Design Button

USABILITY PROBLEMS

Criteria When it hover there supposed to be highlight on the button when it press and click it supposed to be emboss and go to link directly There supposed to be link to correct url address

2

Link

3

Check box

It supposed to give user selection more than one user

4

Option

User supposed to select one out of the option given by web portal

Result Good and according to the design

Good and according to the design Good and according to the design Good and according to the design

• Single usability metrics(SUM) In this metrics we use SUM calculator suggested and created by Jeff Sauro that measured completion rate, errors, task-level satisfaction and test level satisfaction with result as shown in the figure below:

Fig. 5 Single Usability Metrics

Grouping of task from1 to 5 are representing task of usage on 5 different pages. That is Betawi, history, culture, game and quiz. Where we test completion rate, satisfaction, error and time and sum it up using jeff sauro SUM Calculator. The result says that task 1 has the highest SUM percentage of 76%, followed with task 4 70.7%, task 5 69 %, task 3 64.1% and task 2 41.6%. With this result task 1, task 4, task 5 and task 3 have gain good result of usability test (> 60%) which mean the design can be retain. As Task 2 that represent history page have to be redesign as it get low and poor result of usability test (under 60%). Application testing system for web is based Krishen, kota are divided into 6 type which are testing of application usage, testing unit, testing code, HTML, Load testing, user acceptance test and security test. In this research, the conducted test will be: 1. Testing between browsers online by using browsershots.org that states the fastest load time by using google chrome for ± 15 seconds meanwhile Mozilla Firefox and opera ±25 seconds. 2. HTML code test by using online tester on www.validator.w3.org states that the code HTML 5.0 is suitable with the WWW standard. 3. PHP code test with simple test application from simpletest.org states that the php code was made without problems. 4. Compatibility test towards browser and performance application based on HTML 5.0 resulting in optimized application and works well if browser google chrome is used. 5. Loading time testing online at loadimpact.com states that the experiments from various location load time application is more or less 15 seconds. 6. Unit testing is using blackbox testing like shown in table 3 TABLE III.

. BLACKBOX TEST

No

Tests

Criteria

1

main page

System shows page according to the loading time, if browser doesnt have or updated flash player, will be directed to install that plugin.

2

3

4

5

6

7

betawi page

history page

Culture page

Game page

Quiz page

printing results

quiz

Condition after test Good and according to the design

REFERENCES [1]

System shows page according to the loading time, if browser doesnt have or updated flash player, will be directed to install that plugin

Good and according to the design



System shows page according to the loading time, if browser doesn’t have or updated flash player, will be directed to install that plugin




Application shows page according to the loading time, if browser doesnt have or updated flash player, will be directed to install that plugin


Portal shows pop up menu where students’ name, class and school name will be written on before printing result.


[2]

[3] [4] [5]

[6]

[7]

[8] [9]

[10]

[11]

D. Conclusion 1. This research is fourth phase research on Creation of SIPEBI as a cultural edutainment web portal. 2. This research overview user experience using usability metrics to SIPEBI web Portal. 3. This research aim to emphasize of usability metrics measurement of user experience on student accessing SIPEBI. 4. In this research user experience usability metrics is using Single usability metric and usability problems that created by Jeff Sauro. 5. The result it supposed to be redesign since the error are 55.6% higher than 50 %, while satisfaction are 47.7 % below 50% and completion rate is 77.8% below 78% rate.. ACKNOWLEDGMENT Thanks to Annabel for proof reading the journal and all Iconic research students that enrich all material in this paper.

[12]

[13]

[14]

[15]

[16]

[17]

Brooke. J, “SUS - A quick and dirty usability scale”,(Online) http://www.itu.dk/courses/U/E2005/litteratur/sus.pdf accessed 27 June 2014. Garrett. J.J, “The Element of User Experience User centered design for the web and beyond second edition”. New Riders division of Pearson Education, 2011 ,Pg. 6-10. Goni. I. A. M, “Designing of Portlet-based Web Portals”, PHD Disertation University of the Basque Country, Spain,2008, Pg.10 Ifan. K, Iwan P, “Pemanfaatan E-Education untuk meningkatkan Kualitas Belajar”, Proceeding SNATI 2009. UII, Yogyakarta. Itsna. I, “Pembelajaran bahasa arab di mtsn model parakan temanggung perspektif edutainment”, unpublish undergraduate thesis, UIN Sunan Kalijaga Yogyakarta, 2010. Ingham. M, “Interactivity Design in Design sisertation preparation”, (online), http://dd1112.wordpress.com/about/design-dissertationpreparation-2012/, accesed on 15 June 2014 Jirí N., Josef T, “Edutainment or Entertainment Education Possibilities of Didactic Games in Science Education”, ICCP Proceeding BRNO Conference, 2007 (Online).(http://www.iccp-play.org/documents/brno, accessed on 29 December 2013) Katz.N. R, “Web Portals and Higher Education Technologies to Make IT Personal”, Jossey-Bass Inc. 2002, pg. 33. Lausen. H, Stollberg. M, Hernandez. R.L, Ding.Y, Han.S, Fensel.D, “Semantic Web Portals – State of the Art Survey”, Semantic Web Portal Poject, http://sw-portal.deri.org/papers/publications/ SemanticWebPortalSurvey.pdf, accessed on 1 May 2014, pp. 1. Oka A.S., Widya. C, Ilman. Z,. “Perancangan sistem edutainment berbasis web di sekolah menengah atas negeri 1 curup kabupaten rejang lebong”, Informatics engineering journal, Bina Darma e-journal, 2013 (Online), (http://eprints.binadarma.ac.id/view/creators/Saputra=3AOka_Albino=3 A=3A.default.html, Accessed on 29 December 2013) Sauro. J, “10 Essential usability metric”,(online), http://www.measuringusability.com/blog/essential-metrics.php, accessed on 27 June 2014 Saputra. D, Tjhin. Santo , Zufri. T “SIPEBI: An overview of cultural edutainment in web portal as a medium for students’ self-directed learning and alternative’s assessment ”, Proceeding journal 2nd ICEL, Universitas Bandar Lampung, Indonesia 2014, pp. 43-48. Sidiq. W, “Pembelajaran bahasa Arab berbasis edutainment (model ustadz Mohamad Nuaim pada kelas X di MAN Wonokromo Pleret Bantul)”, unpublished undergraduate thesis, UIN Sunan Kalijaga, Yogyakarta, 2010. Utari. S, “Kemandirian belajar: apa, mengapa, dan bagaimana dikembangkan pada peserta didik”. Graduate school UPI math journal, 2010 (Online). (http://math.sps.upi.edu/?cat=3, Accessed on 29 December 2013) Willyanto, “Perancangan Aplikasi E-Education Berbasis Web Dengan Menggunakan Bahasa PHP”. Bina Darma University journal, Palembang, 2011. Zainal. H, “Pengertian web portal”, www.zainalhakim.web.id/pengertian-web-portal.html (Online), Accessed on 29 January 2014. Zufri. T, Saputra. D, Tjhin. Santo, “Desain Portal Edutainment Budaya Indonesia sebagai sarana self-directed learning siswa di tingkat sekolah dasar”, Proceeding journal Sentika 2014, Universitas Atmajaya, Yogyakarta, 2014, pp.484-491.