E-book-based Learning Analytics for Improving Learning Materials Kousuke Mouri, Tokyo University of Agriculture and Technology Department of Computer and Information Sciences Japan
[email protected]
Abstract—The purpose of this paper is to mine or detect meaningful information for improving learning materials using e-book-based logs. The analysis and visualization methods of this study use association analysis with Apriori algorithm. Logs for ebook-based learning analytics were collected in an e-book system. In order to analyze and visualize the collected logs, this paper proposes a system called SILM (System for Improving Learning Materials). Using SILM, instructors or teachers can find the learning materials that should be improved. We believe that improving learning materials with analyzing and visualizing educational big data lead to improving student results. Keywords—learning analytics; educational data mining; ebook; association analysis
I. INTRODUCTION In recent years, the online learning tools such as OER (Open Educational Re-sources) and MOOCs (Massive Open Online Courses) aimed at unlimited participation and open accesses have been spread over the world [1]. Also, many educational institutions have introduced e-learning systems into universities and corporate trainings. In this kind of situations, the large amount of learning logs such as reading and writing the learning materials and taking a memo by using e-book have been accumulated in the data warehouses. As Japanese government policies, they plan to introduce ebooks in all K12 schools by 2020 [2]. Many countries’ policies also focus on introducing the technology of e-books into K12 schools [3], [4]. However, little attention has been paid to visualizing and analyzing the e-book logs in order to improve the quality of teaching and learning. One of its most important issues is how to visualize and analyze e-book logs for improving learning materials. This paper calls visualizing and analyzing e-book logs “E-book-based Learning Analytics (ELA)”. In such analytics, some researchers at Kyushu University and Kobe University in Japan [5], [6], [7] reported that analyzing and visualizing educational big data collected by e-book with LMS (Learning Management System) have a potential for achieving the following goals: (1) Discovering meaningful or interesting learning patterns for education (2) Detecting student’s comprehensive level (3) Predicting students’ final grades
Chengjiu Yin, Kobe University Department of Information Science and Technology Center Japan
[email protected]
(4) Recommending appropriate learning logs and materials in accordance with personalization. We believe that analyzing and visualizing educational big data lead to improving teaching and learning To tackle the issues, this paper proposed ELA for enhancing teaching and learning with improving learning materials. Based on the proposal, we have developed a system called SILM (System for Improving Learning Materials). The system enables teachers or instructors to find the learning materials that should be improved by reasons of various problems such as “it is hard for students to read small letters and small figures” and “It is difficult to understand the e-book content because the order of the page is bad”. The first challenge of this paper is to realize the processes for modifying the uploaded learning materials through human judgement after analyzing and visualizing e-book logs. The rest of this paper is constructed as follows. Section 2 describes related works concerning e-book, learning analytics and educational data mining. Section 3 describes the model for ELA. Section 4 describes system interface we proposed. Finally, this paper describes our conclusion and future works.
II. RELATED WORKS A. Digital or electtronic textbok for education As a genetic term, an electronic book (variously: e-book, eBook, e-Book, ebook, digital book or e-edition) or a digital book is a book-publication in digital form, consisting of text, images, or both, readable on computers or other electronic devices [12]. According to [5], [6] and [7], they introduced ebook system with LMS into university education. If the logs of e-book learning activities are accumulated into the server, then educational big data can be accumulated. They also suggested that analyzing and visualizing the educational big data lead to supporting and enhancing student’ learning activities. However, majority of them didn’t focus on improving teaching and learning by improving learning materials based on the educational big data. To date, many instructor designers have been modifying the learning materials based on their own experiences and students’ qustionnaires, but it is necessary to consider how to modify them based on not only questionnaires
but also learning logs. Our proposed SILM notifies instructor designers or teachers of the learning materials that should be improved, by analyzing and visualizing educational big data. After that, through human judgement, the learning materials can be improved.
B. Learning Analytics Learning Analytics (LA) is measurement, collection, analysis and reporting of data about learners and their contexts, for purpose of understanding and optimizing learning and the environments, which it occurs [8]. The biggest benefits can be pursued through the discovery and understanding of the data’s hidden information. Based on LA, many educational institutions have been increasingly in analyzing available datasets in order to improve student results and learning qualities. It is necessary to consider how the logs collected by e-book system with LMS can be used to improve them.
SCROLL provides an easy-to-use interface. It adopts an approach to share contents with other users based on a LORE (Log-Organize-Recall-Evaluate) model proposed. In addition, they believes that visualizing and analyzing them collected by SCROLL lead to enhancing students’ learning activities in an informal setting. Our approach focuses on how to mine learning materials that should be improved.
III. MODEL FOR E-BOOK-BASED LEARNING ANALYTICS This research objective of this paper is to propose the ebook-based learning analytics model for improving learning materials. Figure 1 shows the learning analytics model for ebook logs. Fig. 1. Learning analytics model for e-book logs
For example, Dashboard is one of visualization ways that students can be easily understood their learning progress and results [15], [17], [23]. According to [9], they reported that information visualization such as network graph based on graph theory [16], 3D representation and heat-map is often more effective than plain text or data. Similarly, some researchers focus on improving students’ results by predicting their future results such as report, mid-semester test and endof-term test scores [5], [6], [7]. We believe that visualization for LA is useful ways in order to understand their own learning progress, results and improvement points. The common idea of those analysis and visualization methods is to intervene students who should be supported, but our system is to improve teaching and learning by improving learn-ng materials.
C. Educational Data Mining Unlike LA, Educational Data Mining (EDM) mainly refers to techniques such as clustering, Bayesian modeling, relationship mining, discovery with models, and visualization. LA considerably focuses on leveraging human judgment, but EDM considerably focuses on automated discovery [10]. For example, association analysis is one of popular data mining in order to detect or mine regularities between variables in large databases [11]. According to [13], they reported that it can be found association rules by grouping students who are enrolled in online education system called LON-CAPA based on parameters such as GPA (Grade Point Average), age, and gender. On the other hand, Mouri et al., [14] use association analysis for mining useful learning patterns from learning logs accumulated in ubiquitous learning system called SCROLL. The objective of SCROLL is to support international students to learn Japanese language in their daiy life for Carrier Support [18] and Onomatopoeia learning [19]. In the SCROLL project, Ubiquitous Learning Log (ULL) is defined as a digital record of what learners have learned in their daily lives [21], [22]. To simplify the process of capturing learners’ learning experiences,
• Data Collection: Generally, at the data collection stage, the logs will be collected from Learning Management System (LMS) called Moodle and e-book system. By using LMS with e-book, operation logs of each student can be collected. • Data Cleansing: It is unavoidable to have repetitive or invalid data among those obtained from data collection, therefore the sorting and cleaning process is needed. According to [13], the data cleansing is especially required when integrating heterogeneous data. Most of researchers in the Data Cleansing described [14], [15] that it is necessary to do this process if analyzing and visualizing big data because duplicated or missing information will product incorrect or misleading statistics (”garbage in, garbage out”). By realizing possible this process, the speed of visualizing and analyzing educational big data collected by systems can be improved.
• Data Analytics: Analytics is carried out based on perspective of LA and EDM. The objectives of this analytics are implementation of system to detect or mine the learning materials that should be improved using association analysis.
While a student goes to previous page, the student will click button, and the operation name will be saved as PREV. While a student want to do bookmark in the page, the student will click bookmark button, and the operation name will be saved as BOOKMARK.
• Data Application: In this stage, in order to provide the analysis results, we developed SILM as described in Section 1. By using the system, instructor designers or teachers could find points for revision of the learning materials uploaded by them. Then, they can grasp the matter based on the analysis results.
It is expected that analyzing and visualizing these e-book logs lead to improve teaching and learning materials because it may be able to find the learning materials that should be improved by reasons of various problems such as “it is hard for students to read small letters and small figures” and “It is difficult to understand the e-book content because the order of the page is bad”. Fig. 2. Interface of selecting digtal learning material in the e-book system
All the above processes can be supported by SILM. By realizing the process, instructor designers can judge whether the learning materials should be improved. In addition, they can discuss and share the information each other. Implementation
A. E-book system and operation logs Most professors are using paper-based books for thir courses, which are distributed by various publishing companies. Thus, it is difficult to make these books available on the e-book system due to copyright issues. We are thus currently using the lecture slides (PowerPoint or PDF) that are created by the teachers and instructor designers. Our developed e-book system coverts the lecture slides into an original e-book file. Students can read learning materials from the ebook system (Fig. 2). Then, they choose the digital learning material on the e-book sytem in order to read it in the viewer. By using the viewer, students can use some functions such as next page, previous page, zoom-in, zoom-out, bookmark, underline, and annota-tion as shown in Fig 3. Our developed e-book system is developed based on HTML 5 technoogy, and students can read digital learning material on the web browser anytime and anywhere. Fig 4 shows sample e-book logs [20]. There are many types of operations in e-book logs. For example, NEXT means that a student clicked the next button to move to the subsequent page. Fig. 4. Sample of e-book logs
Fig. 3. Interface of e-book viewer
page number 1 in the learning material entitled historical source-08. There is a possibility that it is hard for students to read small letters and small figures. Please modify them.”. By referring to them, the instructors can modify the learning material through their own judgement with the analysis results Fig. 5. Visualization interface of SILM
B. SILM: System for Improving Learning Materials SILM is a web application and it is programmed using PHP and Java based on Moodle plugins. Fig 5 shows the visualization interface of the heat-map of SILM. The x-axis shows confidence of association analysis from 0 to 1.0 (It is divided at 0.1).
Fig. 6. Interface for improving Learning materials in SILM
The association analysis of this paper was conducted the following those criteria: Support ≧0.3, Confidence > 0, Lift ≧ 1.0. The objective of the setting value is to detect or mine reliable association rules as far as possible. The y-axis shows each learning behavior: memo, marker, bookmark, zoom-in and zoom-out. After class, instructor designers or teachers firstly check the visualization and analysis results and then they are able to judge whether their own learning materials should be improved. In this study, we use sample data of e-book system in order to analyze and visualize. As shown in Fig 5, if the confidence of zoom-in and zoom-out actions is from 0.9 to 1.0, the number of association rules of the zoom-in is 86 and the number of association rules of the zoom-out is 56. This means that it is hard for students to read small letters and small figures in the learning material because the number of the detected association rules is large. Hereby, the instructor designers can be judged whether the learning material should be improved. As the next step, the instructors need to consider how to modify the learning materials from the analysis results. Fig 6 shows the recommendation for improving the learning material. These recommendations are listed in the ascending order by confidence value. The instructors can learn the learning material improvement know-how based on the recommendations. In this case of Fig 5, the improvement know-how show “Many students frequently used functions of zoom-in and zoom-out with respect to the
IV. CONCLUSION This paper describes how to visualize and analyze e-book logs for improving teaching and learning with improving learning materials. In order to mine or detect the meaningful patterns for improving learning materials, this paper uses association analysis with Apriori algorithm. The analysis was conducted to find the relationships between e-book logs. In order to provide the analysis results, we developed a system called SILM that can be found learning materials that should
be improved. The system enabled instructors or teachers to judge whether their learning materials should be improved based on the analysis results.
[8]
In the future works, we will consider to evaluate SILM in terms of “whether recommendation for improving the learning material are useful information for instructors or teachers” and “whether teaching and learning can be improved by using SILM ”.
[10]
[9]
[11]
ACKNOWLEDGMENT The part of this research work was supported by the Grantin-Aid for Scientific Research No.16H03078 and No. 17K12947 from the Ministry of Education, Culture, Sports, Science and Technology (MEXT) in Japan.
[12] [13]
[14]
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
A. Bozkurt, E. Akgun-Ozbek, S. Onrat-Yilmazel, E. Erdogdu, H. Ucar, E. Guler, S. Sezgin, A. Karadeniz, N. Sen-Ersoy, N. Goksel-Canbek, G. Dincer, S. Ari, C. Aydin, Trends in distance education research: A content analysis of journals 2009-2013. In The International Review of Research in Open and Distributed Learning,16(1).Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1953, 2015 MEXT, Japanese Ministry of Education, Culture, Sports, Science and Technology, “The Vision for ICT in Education”, http://www.mext.go.jp/b_menu/houdou/23/04/__icsFiles/afieldfile/20 12/08/03/1305484_14_1.pdf H. Fang, P. Liu and R. Huang, “The Research on E-book-oriented Mobile Learning System Environment Application and Its tendency”, International Conference on Computer Science and Education, 13331338, 2011. J. A. Shin, “Analysis on the digital textbook’s different effectiveness by characteristics of learner”, International Journal of Education and Learning, Vol.1, No.2, 23-38, 2012. C. Yin, F. Okubo, A. Shimada, M. Oi, S. Hirokawa, M. Yamada, K. Kojima, H. Ogata, ” Analyzing the Features of Learning Behaviors of Students using e-Books”, International Conference on Computer in Education (ICCE 2015), 617-626, 2015. C. Yin, F. Okubo, A. Shimada, M. Oi, S. Hirokawa, M. Yamada, K. Kojima and H. Ogata, “Identifying and Analyzing the Learning Behaviors of Students using e‐Books”, Proc. of 22nd International Conference on Computers in Education 2015 (ICCE 2015), pp.118-120, 2015. K. Mouri, C. Yin, F. Okubo, A. Shimada and H. Ogata, “Profiling Highachieving students using E-book-based Logs”, the first international workshop on Learning Analytics and Knowledge (LAK 16), pp.1-6, 2016.
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
B. S., Simon. (2012). Learning Analytics Policy Brief, http://iite.unesco.org/pics/publications/en/files/3214711.pdf Mazza, R. (2009). Introduction to Information Visualization. SpringerVerlag, London, 2009. George, Siemens, R.S.J.D. Baker, Learning analytics and educational data mining: toward communication and collaboration, proc. of the 2nd International Conference on Learning Analytics and Knowledge, 252254, 2012 R. Agrawal, T. Imielinski and A. Swami, Mining association rules between sets of items in large databases, Proceeding of the 1993 ACM SIGMOD international conference on Management of data, 207-216, 1993. https://en.wikipedia.org/wiki/E-book#cite_note-Interactive_ebook-1 M. Behrouz, K. Gerd, and P. Williams, Association analysis for an online education system, Proceedings of the 2004 IEEE International Conference, 504-509, 2004. K. Mouri, H. Ogata, N. Uosaki, E. Lkhagvasuren, “Context-aware and Personalization Method based on Ubiquitous Learning Analytics”, Journal of Universal Computer Science, pp.1380-1397, 2016. E. Lkhagvasuren, K. Matsuura, K. Mouri and H. Ogata, Learning Log Dashboard: to see your own progress, proc. of the 22nd International Conference on Computers in Education (ICCE 2014), 539-544, 2014. K. Mouri and H. Ogata, Ubiquitous Learning Analytics in the Realworld Language Learning, Smart Learning Environments, Vol.2, No.15, 1-18, 2015. E. Lkhagvasuren, K. Matsuura, K. Mouri, H. Ogata, “Dashboard for analyzing ubiquitous learning log”, International Journal of Distance Education Technologies, 14(3), pp.1-20. N. Uosaki, H. Ogata, K.Mouri and E. Lkhagvasuren, “Career Support for International Students in Japan Using Ubiquitous Learning Log System”, The 15th IEEE International Conference on Advanced Learning Technologies (ICALT2015), pp.78-82, 2015. N. Uosaki, M. Kiyota, K. Mouri, H. Ogata, M. Choyekh, “Onomatopoeia Learning Support for Japanese Learners Using Learning Log System with eBook”, The 16th IEEE International Conference on Advanced Learning Technologies (ICALT 2016), pp.347-348, 2016. M. Kiyota, K. Mouri, N. Uosaki and H. Ogata, “AETEL: Supporting Seamless Learning and Learning Log Recording with e-Book System”, Proc. Of the 24th International Conference on Computers in Education 2016 (ICCE 2016), pp. 380-385, 2016. H. Ogata, B. Hou, N. Uosaki, K. Mouri and S. Liu, “Ubiquitous learning project using life-logging technology in Japan”, Educational Technology & Society, Vol.17, No.2, pp.85-100, 2014. M. Li, H. Ogata, B. Hou, N. Uosaki and K. Mouri, “Context-aware and Prsonalization Method in Ubiquitous Learning Log System”, Educational Technology & Society, Vol.16, No.3, pp.362-373, 2013. E. Lkhagvasuren, K. Matsuura, K. Mouri, H. Ogata, “Learning log dashboard: to see your own progress”, International Conference on Computer in Education (ICCE 2014), 528-532, 2014.
