IC-HUSO 2017

Proceedings of 14th International Conference on Humanities and Social Sciences 2018 (IC-HUSO 2018) 22nd-23rd November 2018, Faculty of Humanities and Social Sciences, Khon Kaen University, Thailand

Investigating Cultural Differences in Online Learning Platform

Patiphan Pholmat1 and Chaklam Silpasuwanchai2 1,2

Information Technology Program, Faculty of Business and Technology 1,2 Stamford International University, Thailand 1 E-mail: [email protected], 2E-mail: [email protected]

Abstract

Online learning platform has attracted lot of interest, with a promise to facilitate and enhance classroom learning. However, little is known how cultural differences affect online learning behaviors. In this work, we investigated online learning behaviors from 233 students using the BlackBoard platform. Descriptive and inferential statistics were performed crossing student’s metadata and online learning behaviors. We found significant effect of cultures and student performance on online learning behaviors. The implications were discussed. This work can provide a foundation for lecturers and academia in using online learning platform to facilitate in-class lecture learning.

Keywords: Cultural Differences, Online Learning, Blackboard

Paper Number: ICHUSO-071

298


1. Introduction Online learning platform such as Moodle or Blackboard has proven useful to facilitate classroom learning (Kauffman, 2015). Such platforms reduce the cost of learning resources, enable self-paced and self-directed learning, facilitate different learning styles, and is intuitive for this generation of students who are digital natives (Bartholomew, 2017; Rashid & Asghar, 2016). Consequently, many institutions adopted blended learning pedagogy which defines an learning approach that combines online digital media with traditional classroom methods. Such approach has been found to promote self-directed learning as students are in better control of the time, place, and pace of learning (El-Seoud et al., 2017; Sun & Chen, 2016) and has also been found useful to facilitate new learning concept such as flipped classroom (Kong, 2014). However, little is known how cultural differences interact with such online learning platform. It is well-known that cultural differences play a big role in shaping one’s learning styles. For example, Asian students were likely more passive in discussion groups and inquiry during lectures (as contrasted with Western students) (Lu et al, 2010; Manikutty, Anuradha & Hansen, 2007). Given such strong evidence, we seek to build upon past work and investigate how cultural differences relate with online learning platform. For instance, is there any particular culture that favors or feel uncomfortable toward online learning platforms? How lecturers should best utilize online learning platforms in a “one-culture” or “cross-culture” classrooms? In addition, given that we are going to analyze the data anyway, it is tempting to ask other related question, for example, aside from culture, is there any other interesting “user” variables that also influence the use of online learning platforms, e.g., students’ area of study, or student’s GPA? Our work aims to touch on these questions. In this paper, we took a typical statistical approach to investigate our aforementioned questions. Statistical approach combines descriptive and inferential statistics amongst others to provide instructors and learners with a better understanding and insight on learning and teaching performance (Pechenizkiy, 2017). Our own view is that quantitative method fits our study as plenty of digital footprints (e.g., log data) and user metadata can be analyzed via online learning platform. Specifically, in our work, we have performed ANOVA and PostHoc with Bonferroni Correction tests to understand the relationships between cultures (as well as other user characteristics) and the use of online learning platforms. Our results found a significant effect of cultures and student performance on the use of online learning platform. Interestingly, Asian tend to be more involved in online learning platform, as compared to their Western friends. In addition, we found a difference in the use of online learning platform across disciplines. Specifically, we found that students from Information Technology discipline tend to use online learning platform more than other disciplines. These counterintuitive results have been discussed in the discussion section. 2 Literature Review Online learning platform has been often discussed regarding its usefulness to enhance classroom learning (Bartholomew, 2017; Rashid & Asghar, 2016). Online learning platform, e.g. Blackboard, and Moodle, can connect students outside of the classroom for supporting their study in blended learning pedagogy. Instructors also can take advantage of its capabilities to deliver course contents, organize assignments and grades, and manage discussion boards (Kauffman, 2015; McGee & Reis, Paper Number: ICHUSO-071

299


2012). Moreover, studies revealed significant relationship between students and online participation (Alzahrani, 2017). One of common topics being researched is regarding online learning analytics, focusing on using statistical and predictive approach to analyze students’ behaviors (Sclater, Peasgood, & Mullan, 2017). Every student interacts with their learning management system or completing online assignments. Thus, they leave huge amount of digital footprint and log data. Such quantitative approach also provides certain advantages over the traditional evaluation which tend to focus only on using subjective measures such as surveys or interviews which, if not appropriately employed, could result in biased results (Jara et al., 2017). Given this advantage, this quantitative approach on online learning platform has been practiced by many researchers. Whitmer (2012) examined online learning behaviors of 377 students on a course which had recently been redesigned to integrate much deeper use of learning technology. He looked at several student characteristics that can potentially influence online learning behaviors including gender, minority racial/ethnic group, income, grades, and whether the student was the first in family to attend challenge. He found that total hits (i.e., online participation) is the strongest predictor of student success. Arnold & Pistilli (2012) and Agnihotri & Ott (2014) have collected many student performance metrics via online learning platforms, such as points earned so far, interaction with the online platform, student characteristics (e.g., age) and fed these metrics into a predictive algorithm in order to identify students who most in need of support with an aim to increase retention rate. They found that key risk factors include grades, subject being studied, student’s certainty in their choice of major, and financial support. Sclater, Peasgood, & Mullan (2017) studied how online usage relates to student performance. Analyzing 131 courses, they found that students who obtain D or lower use online platform around 40% less than those with a higher grade. This implies a strong relationship between online participating and student performance. In this paper, we seek to build upon past work by analyzing student cultures along with other available student characteristics and investigating how these characteristics affect online learning behaviors. Cultures have long been a subject of study on how it influences learning. For example, it is commonly well-known that Asian students are more passive compare to Western student (Corvette, 2007; Swierczek & Bechter, 2010). Nevertheless, there is still a few studies focusing on investigating how cultural differences influence online learning behaviors. Particularly, we aim to investigate how different cultures interact with different components in online learning platform such as discussion forum, video lectures, text lectures, etc. 3 Methodology This article reports an investigation from a study of international university in Thailand that use blended learning pedagogy to support students to develop digital literacy knowledge. A total of 233 students with 3 independent variables (cultures, disciplines, grade) and 3 dependent variables (content view counts, assignment submission counts, and discussion submission counts) were analyzed. 3.1. Classrooms A 13-week teaching was conducted. The teaching covered on a topic in the General Education subject, namely “Computer Applications” which focuses on Digital Literacy. The entire teaching amounted for 2880 mins, with face-to-face teaching amounted 1440 min (12 weeks * 2 hours * 60 Paper Number: ICHUSO-071

300


minutes), and with virtual learning having the same amount of 1440 min, spanning 11 lessons of study across 8 classes. Six different lecturers were responsible for the trial teaching. The years of teaching experience for the five lecturers ranged from one to twenty. Learning materials including slides, supporting videos and references, assignments are posted on the university’s online learning platform, namely Blackboard. Materials were provided at least one week prior to the face-to-face. After the 2 hours of face-to-face session with the lecturer, each student is required to revisit these online learning materials, conduct self-study, and submit the given assignments of that week. Assignments’ deadline was due one week after. Students were allowed to re-submit their assignments until the deadline. Using discussion groups are optional and are not graded. 3.2 Online Learning resources Online learning resources are provided in Blackboard to enable student to revisit course syllabus, review lesson goals, download lecture materials, supporting videos, and reference materials. Assignments were given to student every week with a total of 11 assignments. Assignments, formative assessments, and summative assessments are same across the 8 classes. Discussion forum is available to enable students to discuss knowledge but is optional. Along with Blackboard, Microsoft Office 365 Suite is provided to complement Blackboard, e.g., sharing documents online using OneDrive, sharing online recordings using OfficeMix.

Figure 1: Online learning resources


301


3.3 Method 233 student characteristics and their online learning behaviors were gathered after their final exams. The data spanned the 1st face-to-face class until the last day of final exams, which counted a total of around 90 days (the final exam day was different between classes). The data were extracted through the Blackboard Statistical Report. Student characteristics (independent variables) were gathered which include nationalities, disciplines, and grade. Other metadata were discarded such as Student ID, Name, Class section, etc. For feature engineering, we grouped several nationalities into a regional culture (e.g., Thailand -> Southeast Asia) to prepare our data for cultural analysis. Five final cultures were grouped Africa, East Asia, South Asia, Southeast Asia and Europe and America. Discipline wise, we grouped similar discipline into one category (e.g., Accounting and Finance -> Business Administration). Four broad disciplines were grouped - Information Technology, Communication Arts, Business Administration, and Arts. Grade was grouped into four levels (i.e., A, B+ -> high; B, C+, C -> medium; and D+ and D -> low; F -> fail). Any missing values were filled with mode given our data categorical nature. Online learning behaviors (dependent variables) were collected which include content view counts, assignment submission counts, and discussion submission counts. Content view counts is a measure of how often student visits contents. Contents were defined as any learning materials posted on Blackboard including lecture notes, supporting videos, and reference materials. Assignment submission counts measure how many times each student submit their homework. Since students were allowed to re-submit their assignments until the deadline, the assignment count could be more than 11 (the number of assignments). Discussion submission counts measure how often students participate (visiting/posting) in a discussion forum. Given the discussion is optional, discussion submission count for some student could be zero. 3. Results and Discussion This section reports on the results from our descriptive and inferential statistics analysis. Analysis of Variance (ANOVA) and PostHoc analysis with Bonferroni correction was conducted crossing cultures, disciplines, grades ~ content view counts, assignment submission counts, discussion submission counts. 3.1 Cultural Differences on Online Learning Behavior Table 1: Cultural Differences on Online Learning Behavior

Africa East Asia


Assignments

Course Content

Discussion Board

Mean

S.D.

Mean

S.D.

Mean

S.D.

10.70

2.79

143.52

55.95

0.57

0.99

9.84

4.84

162.06

91.51

0.64

1.53

302


Europe and America South Asia Southeast Asia

9.50

4.12

87.30

32.17

0.33

0.89

12.37

3.04

156.89

77.47

0.74

1.24

10.77

4.02

132.58

61.97

1.13

1.72

Table 1 shows the means and standard deviations of the results. One-way ANOVA test shows significant difference between cultures on content view counts (F(2, 228) = 3.12; p = 0.015 ). A further posthoc analysis with Bonferroni correction confirms the difference between Africa and Europe and America, between South Asia and Europe and America, and between East Asia and Europe and America (all p < 0.01 after bonferroni correction). We found no significant effect of cultures on assignment submission counts (p = 0.22) and discussion submission counts (p = 0.084). Our result confirms the effect of cultural differences on online learning behavior. Particularly, we found that cultures do influence content view counts. It is very tempting to speculate how culture relate with content view counts. One explanation could be due to the digital proficiency of certain cultures. However, to our surprise, Europe and America has the lowest content view counts among the others, which contradicts Swierczek, F.W. & Bechter, C. (2010) and Lu, J., et al. (2010) as they found European were more active learners compare to Asian learners. Another possible explanation is due to the nature of our content that may not be preferable by Europe and America. This is very plausible because the contents were mostly agreed upon the lecturers where all 6 lecturers are from Asia. Thus, the content may tailor more toward Asians. Further investigation (possibly interviews) should be conducted to confirm this. 3.2 Disciplines on Online Learning Behavior Table 2: Disciplines on Online Learning Behavior Assignments

Course Content

Discussion Board

Mean

S.D.

Mean

S.D.

Mean

S.D.

Information Technology

9.58

4.54

139.92

101.45

1.67

2.23

Communication Arts

8.94

4.09

106.31

46.69

0.88

1.36

Business Administration

11.02

3.85

141.15

68.13

0.88

1.51

Arts

10.09

4.50

135.14

59.49

0.95

1.43

Table 2 shows the means and standard deviations. One-way ANOVA test found no significant effect of disciplines on content view counts (0.19), assignment submission counts (p = 0.15) and discussion submission counts (p = 0.431). The lack of differences could be due to the large variability of students.


303


Although we did not find any statistical difference, it remains useful to look at some interesting differences in means between disciplines. According to the data, a noticeable difference can be observed in Content and Discussion. In Content, Communication Arts has the lowest content view counts with a sizeable difference compared to other disciplines. Possibly, Comm. Arts students may prefer to learn using different learning styles, e.g., visual learning styles (e.g. lecture video, picture) (Manikutty, Anuradha & Hansen, 2007). Although our Blackboard has substantial amount of visual aids, it may prove inadequate. Further investigation should be conducted on learning styles and material types to confirm this premise. In Discussion board, Information Technology has a sizeable lead compared to other disciplines. This could be due to (1) the relatively higher digital competency of IT students, and (2) the adeptness of IT students in discussing online. 3.3 Grade on Online Learning Behavior Table 3: Grade on Online Learning Behavior Assignments

Content View

Discussion Board

Mean

S.D.

Mean

S.D.

Mean

S.D.

High

12.36

2.71

149.38

44.81

1.07

1.66

Medium

11.01

3.03

148.99

71.27

0.80

1.45

Low

9.15

3.73

115.64

71.23

0.93

1.57

Fail

3.33

2.72

74.69

50.97

0.43

0.73

Table 3 shows the means and standard deviations. One-way ANOVA test found significant effect of Grade on content view counts (F(9, 223) = 4.15; p < 0.001 ). A further posthoc analysis with Bonferroni correction confirms the difference between High, Med, Low with Fail (all p < 0.0125 after correction). Similarly, a significant effect of Grade on assignment submission counts (F(3, 229) = 11.39; p < 0.001) was found. A further posthoc analysis with Bonferroni correction confirms the difference between all pairs (all p < 0.0125 after correction). Last, ANOVA found a significant effect of Grade on discussion submission counts (F(3, 229) = 2.82; p < 0.05). However, a posthoc analysis with Bonferroni correction cannot confirm any difference between any pairs (all p > 0.0125 after correction). This grade analysis confirms past work findings (Davies & Graff, 2005) which indicate that student performance (grade) and online learning behavior are strongly related. In terms of content view counts, assignment submission counts, and discussion submission counts, all are linearly related with High, Medium, Low and especially for Fail. 4. Conclusion Our work studied how cultural differences affect online learning behaviors, along with other user characteristics. This study empirically confirms the effect of cultures as well as reconfirming the effect of student performance. Our study also suggests a minor effect of disciplines which could arise due to digital proficiency and preferences.


304


Our work has three distinct implications toward the use of online learning platform: (1) it is important to employ cultural sensitive instructional design in an international online learning environment. In our study, we found that Europe and America students have the lowest content view counts, which contrasts past study, which is likely due to the “Asian” design of our online materials, (2) it is important to account for digital proficiency and preferences in different disciplines when designing online learning materials. Certain disciplines may be more adept than other disciplines, and vice versa. Providing enough training and incentives are important for certain disciplines to use online learning platform effectively, (3). our study, along with past work, strongly suggests the use of online learning platform, as the results clearly shows a strong relationship between student performance and online learning behaviors. Certain limitations of our work can be discussed. First, we have been primarily focusing on Digital Literacy class. Our future work aims to analyze similar data on other courses. Second, although we have fairly large sample, our inferential method would benefit from a larger sample, as seen in the variability of the sample. Third, our study employed mainly quantitative method. Future work should attempt a mixed method, combining qualitative method to grind further deep insights regarding student behaviors. 5. References Alzahrani, M. G. (2017). The Effect of Using Online Discussion Forums on Students’ Learning. The Turkish Online Journal of Educational Technology, 16(1), 164-176. Agnihotri, L. & Ott, A. (2014). Building a Student At-Risk Model: An End-to-End Perspective From User to Data Scientist. Proceedings of the 7th International Conference on Educational Data Mining. London, UK: International Educational Data Mining Society. Arnold, K. E. (2012). Course Signals at Purdue: Using Learning Analytics to Increase Student Success. In S. B. Shum, D. Gasevic, & R. (eds.) Ferguson. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. Vancouver, BC, Canada: Society for Learning Analytics Research. Agnihotri, L. & Ott, A. (2014). Building a Student At-Risk Model: An End-to-End Perspective. In Stamper, J., Pardos, Z., Mavrikis, M., McLaren, B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining. Bartholomew, S. R. (2017). Relationships Between Access to Mobile Devices, Student Self-Directed Learning, and Achievement. Journal of Technology Education, 29(1), 2-24. doi: 10.21061/jte.v29i1.a.1 Davies J. & Graff M. (2005). Performance in e-learning: online participation and student grades. British Journal of Educational Technology, 36(4), 657-663. El-Seoud, M. S. A., & et al. (2017). E-learning and students' motivation: A research study on the effect of e-learning on higher education. International Journal of Emerging Technologies in Learning, 9(4), 20-26. doi: 10.3991/ijet.v9i4.3465 Fritz, J. (2013). Using analytics at UMBC: Encouraging student responsibility and identifying effective course designs. Retrieved July 18, 2018 from https://library.educause.edu/~/ media/files/library/2013/4/erb1304-pdf.pdf Jara, M., & et al. (2017). Patterns of library use by undergraduate students in a Chilean University. Libraries and the Academy, 17(3), 595-615. Jayaprakash, S. M., & et al. (2014). Early alert of academically at‐risk students: An open source analytics initiative. Journal of Learning Analytics, 1, 6–47. Paper Number: ICHUSO-071

305


Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 1-13. doi: http://dx.doi.org/10.340 2/rlt.v23.26507 Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy. Computers & Education, 78, 160-173. doi: 10.1016/j.compedu.2014.05.009 Lu, J., et al. (2010). Cross-cultural education: Learning methodology and behaviour analysis for Asian students in IT field of Australian universities. In T. Clear & J. Hamer. (Eds.). Proceeding of 12th Australasian Computing Education Conference (ACE 2010). Brisbane, Australia: Australian Computer Society, 117-126. Manikutty, S., Anuradha, N.S., & Hansen, K. (2007). Does culture influence learning styles in higher education?. International Journal of Learning and Change, 2,(1). doi: 10.1504/ IJLC.2007.014896 McGee, P., & Reis, A. (2012). Blended course design: A synthesis of best practices. Journal of Asynchronous Learning Networks, 16(4), 7-22. Pechenizkiy, M. (2017). From the president of the international educational data mining society. In C. Lang, et al. (Eds.). Handbook of learning analytics. doi: 10.18608/hla17. Rashid, T. & Asghar, H. M. (2016). Technology use, self-directed learning, student engagement and academic performance: Examining the interrelations. Computers in Human Behavior, 63, 604-612. doi: 10.1016/j.chb.2016.05.084 Sclater, N., Peasgood, A., & Mullan, J. (2017). Learning analytics in higher education: A review of UK and international practice. Retrieved July 2018, from https://www.jisc.ac.uk/sites/ default/files/learning-analytics-in-he-v3.pdf Sun, A., & Chen, X. (2016). Online education and its effective practice: A research review. Journal of Information Technology Education, 15, 157-190. doi: 10.28945/3502 Swierczek, F.W. & Bechter, C. (2010). Cultural features of e-learning: A euro-asian case study. In Spector, J.M., et al. (Eds.). Learning and instruction in the digital age. New York: Springer. doi: 10.1007/978-1-4419-1551-1_18 Whitmer J. C. (2014). Logging On to Improve Achievement: Evaluating the Relationship between Use of the Learning Management System, Student Characteristics, and Academic Achievement in a Hybrid Large Enrollment Undergraduate Course. Dissertation in Doctor of Education. Oakland, California: Office of Graduate Studies, University of California.


306