Increasing Undergraduate Students' Exam

TOJET: The Turkish Online Journal of Educational Technology – July 2016, Special Issue for IETC, ITEC, IDEC, ITICAM 2016

Increasing Undergraduate Students' Exam Performances In Statistics Course Using Software Socrative Nuri BALTA Faculty of Education, Canik Basari University, Samsun, Turkey [email protected]

Selim GÜVERCIN Faculty of Natural Sciences, Suleyman Demirel University, Almaty, Kazakhstan [email protected]

Abstract Socrative is a free online assessment tool that empowers instructors to engage and assess their students as learning happens. Through Socrative, instructors can design assessments online and students can access the assignments with any device or browser. In this study, the Socrative was used to increase students’ exam performances in Canik Basari University- Turkey, in fall semester of 2015-2016 academic year. The paper describes how a consistent use of Socrative facilitated students' exam preparation in an introductory statistics course. A post-test design with control group was deployed to examine the effect exam preparation through Socrative on students’ grades. The ANCOVA results showed that there existed a statistically significant difference between control and experimental group students’ final grades (F(1, 92)= 6.92, p=.010, 2= .07), once their means had been adjusted for midterm grades. Further, results showed that the students are satisfied with this preparation style likely because the immediate feedback given helps to improve student understanding of the concepts. This study indicated that along with usual usage it can also be burdened to increase students’ performances on exams by sharing homework sets through Socrative. Keywords: Software Socrative, Exam preparation, Statistics course, Online homework INTRODUCTION Quick advancement of technology is transforming the educational environment. Schools nowadays are more equipped with smart boards, tablets, Internet, and computers than a decade ago. Along with the technological growth the teaching and assessing approaches also change. For instance, it is now easier to apply a survey, instantly aggregate the results and analyze more quickly than two decades ago. One of the emerging technology is the web based student response system (SRS) which was generally operated with clickers. The use of SRS is becoming more widespread in higher level education (Heaslip, Donovan, & Cullen, 2014). There are several online SRS programs such as Socrative, Edmodo, Weebly, and Class Dojo. This study is dealing with Socrative which is popular among the instructors of Canik Basari University. Socrative software Socrative is a free online assessment tool that empowers instructors to engage and assess their students as learning happens. Through Socrative, instructors can design assessments online and students can access these assignments with any device (laptops, phones, or tablets) or browser. More specifically, it allows instructors to administer in-class surveys, homework assignments and quizzes, aggregate result in real time, generate reports to monitor and visualize student learning (Socrative, 2016). When quizzes are completed, teachers have the option to generate reports to email, save to Google Drive, or download as Excel or portable document format (PDF) files. One of the most beneficial aspect of Socrative platform is that it allows the instructors to mix both the order of questions and the order of their choices. For instance, during a quiz different questions appear on the mobile phone of each student. Moreover, the process of creating and reordering questions is very intuitive. Another, useful application is that, it allows groups of students to work through quizzes in a race to answer the most questions correctly. Where correct answers move an avatar forward and incorrect answers keep it in place. Teachers and students will log in to Socrative software in different way. As such, an instructor must sign up for a teacher account. Upon creating an account, so teachers will assigned in a unique room code. In terms of students, they will use this code to identify the virtual classroom, in this way they can gain access to quizzes, homework or surveys that teachers share. Importantly, logging on to Socrative using the room code does take a few minutes for students.

Copyright © The Turkish Online Journal of Educational Technology

314


Socrative platform allows to create quizzes and homework sets that can include any combination of multiple choice, true or false, and short-answer questions. Furthermore, it include the ability to attach images to questions as well as to add explanations of correct answers so that students can have immediate feedback. Moreover, the prepared materials can be shared with colleagues and one can import preexisting quizzes into his/her virtual classrooms, allowing collaboration with colleagues. Overall, this user-friendly tool is very convenient. It is highly recommended for those who are looking for a means of assessing student learning as well as increasing classroom engagement (Nawalaniec, 2015). The use of Software Socrative in university teaching is rapidly increasing in contemporary time. It can be used for several aims such as; improving student engagement and the learning experience (Dervan, 2014; Liu & Taylor, 2012), just-in-time teaching (Krause, 2013), active learning (Coca & Slisko, 2013), making the lecture more enjoyable both for the students and for the teacher (Piatek, 2014), collaborative learning (Awedh, Mueen, Zafar, & Manzoor, 2014), and receiving immediate feedback about instruction (Hadiri, n.d.). Web-based homework Homework is traditionally assigned to students by instructors to be completed at home, for students to increase their content knowledge. However, students require feedback after completing any assigned homework. Once instructor’s feedback is executed, students are then able to adjust their errors (Leong, 2014). On the other hand, some students need special supports to be motivated, to study systematically and to prepare thoroughly for the exams. One way to give feedback to students’ homework solutions is web based homework platform, which, in general, refers to any system of homework problems made available online to students with automatic answer grading capability and immediate feedback on correctness of student solutions (Demirci, 2010). A student then retries the question again receiving feedback if an incorrect solution is given. Given the crowd nature of typical statistics classes at the university level and the inability of teachers to provide such feedback consistently to every student in the room, where software that is able to do so, provides a solid support to students and serves to support in-class instruction. In doing problems and assignments with the aid of their hand on technological devices, students are able to practice the material, refine their understandings and arguably improve their ability to master the content. Rapidly growing technological development and widespread use of the Internet in instruction encourage teachers to assign web based homework (Altun, 2008). On the other hand, the use of online homework by instructors has leaded to search on several aspects of web-based homework applications, such as, students attitudes toward online homework (Altun, 2008; Leong, 2014), the effect of web-based homework on students’ achievement (Babaali & Gonzalez, 2015; Chow, 2014; Demirci, 2010), and student perceptions of online homework use (Ongun, Altas, & Demirag, 2011; Richards-Babb, Curtis, Georgieva, & Penn, (2015). This study has further burdened Socrative and used it as a web based homework platform. The aim of this study was to increase students’ performances on exams by sharing homework questions through Socrative. The research described herein, was driven by the following research questions. 1. What affect does the use of Socrative as an online homework system have upon students’ exam grades in the undergraduate statistics course? 2. What is the attitude of the students towards the use of Socrative as an online homework system? METHOD In this study, the Socrative was used to increase students’ exam performances in Canik Basari University (CBU), Turkey. The paper describes how a consistent use of Socrative facilitated students' exam preparation in an introductory statistics course for business, international commercial and business (ICB), and psychological counseling and guidance (PCG) majors. The methods used included both quantitative and qualitative data. A post-test design with control group (Fraenkel, &Wallen, 1996) was deployed to examine the effect exam preparation through Socrative on students’ grades. Qualitative data were collected to identify the strengths and weaknesses of using Socrative as a tool to give homework to enhance students’ exam performances through an online questionnaire. Participants Canik Basari University is a four-year degree granting private institution located in Samsun, Turkey. It is one of two universities in Samsun which has four faculties: Education, economics and administrative sciences, architecture and engineering, and arts and sciences. While the faculty of education offers four-year degrees in areas: Turkish education, foreign languages education and, psychological counseling and guidance; the faculty of economics and administrative sciences offers four-year degrees in areas of: business, international commercial


315


and business, and political science and international relations. The faculties offers an extensive list of courses in these areas. Among these is Statistics-I: The research presented here focuses on this course. Specifically, we look at data collected in the form of final exam scores and analysis of students’ attitudes on using Socrative as a tool for exam preparation. In total, 46 students in their second year in business major, 23 students in their second year in ICB major and 78 students from PCG major at their first year were taking introductory Statistics-I course. This was a course including descriptive statistics along with correlation and regression topics. It is a pre-requisite for Statistics-II course which contains inferential statistics concepts. The course proves to be a challenge to many students who do not have a strong mathematics base. Fifteen, eight and six students respectively from aforementioned majors were repeating the course. Since they were not required to attend the lessons they were excluded from all analysis. Moreover, since 23 students did not indicate their amount of participation to the homework activities were also excluded. Thus, a total of 95 undergraduate students, 58 female and 37 male, were used for all analysis. Table. 1. Demographics of the participants

Gender Major

Variable Men Women Business International commercial and business Psychological counseling and guidance

f 37 58 23 15 57

% 39 61 24 16 60

The intervention In CBU the fall semester of 2015-2016 academic year ended in December. All students were allowed one week to prepare for their final exam. During this break two snowy days elongated the gap to nine days. The instructor (the first researcher of this study) of Statistics-I course shared seven homework sets during the break. The homework sets comprising various types of questions were shared each evening and was replaced with a new one after 24 hours. The homework questions were shared through Socrative which allowed the instructor to see students’ performances on each question. The instructor shared the answer of each homework set after 24 hours and he shared questions similar to those least answered in the previous homework in new homework sets. During the last lesson of the course students were announced for this application and the announcement was shared in students’ email, Twitter and Facebook groups. Besides, Socrative was used in several lessons so that all students were familiar with its usage. Totally seven homework collections were shared and students participated to these homework sets in different percentages. Of the 95 students 32 never participated and 5, 4, 12, 13, 11, 9, 9 students participated 1, 2, 3, 4, 5, 6, and 7 times respectively. Students that never participated and those participated one and two times were taken as control (41 students) and those who participated three or more times were taken as experimental group (54 students). Instruments Three sources were utilized to collect data. First, students’ midterm grades was used to equate the groups’ initial differences. The midterm was a regular exam applied to assess students’ knowledge related to descriptive statistics topics, such as mean, median, mod, standard deviation, quartiles, Box and Whisker Plot, standard normal distribution, and Z and T scores. The midterm consisted of 41 multiple choice questions prepared by the first author from various statistics books, such as “Statistics for the behavioral sciences” (Gravetter, & Wallnau, 2006) and “Statistical Methods for Psychology” (Howell, 2012). Second, students’ final exam grades were used to reveal group differences after the intervention. In other words, some of the students prepared for the exam through Socrative however some of them did not. On the final exam students were asked 8 open ended questions assessing students’ knowledge on correlation and regression at basic level. These questions were also selected from various statistics books such as “Introduction to Statistics and SPSS in Psychology” (Mayers, 2013) and “Statistical Methods for Practice and Research- A Guide to Data Analysis Using SPSS” (Gaur & Gaur, 2009). Third, an online questionnaire of eight Likert type (1 – Strongly Disagree, 5 – Strongly Agree) items and three essay questions to extract information on the students’ judging of the use of Socrative for exam preparation. The questionnaire items (see Table 1) were prepared by the authors. The final version of the questionnaire was decided after several electronic mailing between the authors. Then, the questionnaire was administered to three


316


students in a one-on-one situation to make sure that students interpreted the questions unambiguously. We observed that they answered all items without hesitation and were able to interpret all questions correctly. During the last homework sharing, the students filled out this online questionnaire which was published on Socrative. On the other hand, the internal consistency of the survey questionnaire; Cronbach‘s alpha value obtained was .913. RESULTS The effect of doing homework through Socrative on final exam results were analyzed with ANCOVA where students’ midterm grades were used as covariate. The aim behind using ANCOVA was to equate the preexistence differences between control and experimental group students (Pallant, 2013). The strong linear correlation (.64) between students’ midterm and final grades allowed using it as covariate. The correlation coefficient .64 suggests a fairly predictable relationship between students’ midterm and final grades. Two of the most important assumption of ANCOVA; Levene's test for homogeneity of variances and the homogeneity of regression slopes, were tested. It was found that the error variance of the final grades was equal across the groups (those who participated and who did not participated to homework activities), and no interaction between the covariate and the independent variable (final exam scores) was detected (F(1, 91)= 1.67, p= .20). The descriptive statistics indicated that the mean of final grades of students who did not participate in Socrativeassisted homework activities was 31.51 and for those who participated in the activity was 58.24. The difference between group means was assessed with ANCOVA (Table 1). Table 1. ANCOVA results Source

Type III Sum of Squares

df

Mean Square

F

p

2

Corrected Model

38450.74

2

19225.37

37.85

.000

.451

Intercept

326.95

1

326.95

.644

.424

.007

midterm

21801.10

1

21801.10

42.92

.000

.318

group

3516.69

1

3516.69

6.92

.010

.070

Error

46729.01

92

507.92

Total

292411.00

95

Corrected Total

85179.75

94

The ANCOVA results showed that there existed a statistically significant difference between control and experimental group students’ final grades (F(1, 92)= 6.92, p=.010, 2= .07), once their means had been adjusted for midterm grades (see Table 2). This means that when students’ midterm grades are controlled there is a significant effect of sharing homework sets through Socrative on students’ final grades. Meanwhile the eta square value, .07 indicated a moderate effect size. The moderate effect is based on the Cohen’s (1988) conventions for effect sizes; 2= 0.01 (small), 2= 0.06 (medium), and 2= 0.14 (large) effects. Finally, the adjusted means with the effect of covariates was found 41.14 and 50.43 for control and experimental groups respectively. In other saying, after ANCOVA adjustment, final grades of the control groups increased for 9.63 points while that of experimental group decreased for 7.81points. The results of the attitude survey indicated that students have a positive moderate attitude toward preparing for exam with Socrative application. The maximum average score that could be obtained from the survey was five and average of all students was calculated as 3.81 showing a moderate attitude. Table 2 includes the items of the questionnaire along with mean and standard deviation of each item.


317


Table 2. Descriptive statistics of questionnaire items # 1 2 3 4 5 6 7 8

Items This application serve my purpose I liked Socrative I regularly prepared for the exam with this application With this application I perpared for the exam with relish With Socrative I prepared more for the exam I want this application to be used in the next exam also Socrative should be used in other subjects as well I hope to get a high mark in the exam with this application

Mean

SD

3.92 4.08 3.69 3.56 3.84 4.01 4.06 3.35

0.98 0.95 1.04 1.14 1.02 1.09 1.08 1.06

As seen in Table 2, three items (2, 5 and 7th items) are related to Socrative software and the others (1, 3, 4, 6, 8th items) are assessing the use of Socrative as a tool for homework sharing. While the mean for items that asses Socrative was found 4.00, the mean for the usage of Socrative for sharing homework sets was found as 3.71. Students have scored the item “I liked Socrative” the most (4.08) and the item “I hope to get a high mark in the exam with this application” the least (3.35). Besides the Likert type questions assessing the students’ attitudes toward the use of Socrative as a homework platform, there was a question asking students amount of participation to the homework activities. Seven homework sets were shared within 9 days and students’ participation rates varied between 1 and 7 times with mean and standard deviation of 4.33 and 1.77 respectively. Another positive indicator of using Socrative to prepare students for exam by sharing homework sets is the correlation between students’ final grades and amount of participating to homework question solutions. The Pearson correlation results indicated that there was a significant positive large association (.52) based on Cohen’s (1998) classification (.10 are “small,” those of .30 are “medium,” and those of .50 are “large”). The last two items on the questionnaire were “What was the best use of sharing homework questions on Socrative?” and “What are the problems associated with sharing homework questions on Socrative?” To represent the students’ responses we constructed two answer sculptures as shown in Figure 1 and Figure 2. In the answer sculptures rounded rectangles denote responses related to use of Socrative as homework sharing platform, trapezoid ones represent responses directly related to Socrative software and the elliptic ones symbolize other types of responses.

Seeing teacher’s exam style (4)

Preparing in detail (2)

Reviewing the topics (13)

Solving different types of questions (3)

What was the best use of sharing homework questions on Socrative?

No response (23)

Easy access and practical use (8)

Encourages to study more (3)

Proving for the exam (2) Providing feedback from teacher (5) Preparing easefully at home (2)

Seeing sample questions (20)

Allow to systematically preparing for the exam (8)

Figure 1. Answer sculpture for best use of Socrative as a homework sharing platform


318


As seen in Figure 1 %25 of the responses are directly related to proving the Socrative. For example, 8 students found it useful in allowing to systematically preparing for the exam and 3 students declared that it encourage them to study more. Similarly, %24 of the students did not supplied any responses and %48.4 of the students supplied responses related to best use of Socrative as a homework sharing platform. For instance, 20 students found that platform is useful, where it allows seeing sample questions, and 2 students found it helpful by allowing to calmly studying at home. The answer of questions should be supplied immediately (8)

The answers should be provided on time (1)

No problems (15)

Lecture notes should be attached (1)

Having questions on paper is better (2)

We also want to see the rate of correct answers (2)

No response (58)

It is proper for multiple choice problems (1)

What are the problems associated with sharing homework questions on Socrative?

I sometimes cannot login to Socrative (1)

It is not a good software (1)

We cannot access the past homework sets (3)

The solution of the questions should be given (2)

Figure 2. Answer sculpture for the problems associated with the use of Socrative as a homework sharing platform As seen in Figure 2 %61 of the students did not supplied a response for the problems associated with the use of Socrative for sharing homework sets and %16 of them found no problems. However, %23 of the students stated various problems. For instance, the fact that students could not immediately get the answers of the homework sets (8 students) and instead of on paper, having question on their tools’ screen (2 students) were seen as problems. Similarly, could not accessing the past homework sets (3 students) and sometimes could not logging to Socrative (one student) was the problems directly related to Socrative that students mentioned. DISCUSSION and CONCLUSION As is evidenced by the data presented, those students in the treatment group had a higher average on the final exam than those in the control group. In other words, participants having the Socrative-assisted homework before the final exam performed well within the range of those without the Socrative-assisted homework. This conclusion do agree with those of (Altun, 2008) who indicated that online assignments increased student motivation and academic achievement, and those of (Babaali & Gonzalez, 2015) who showed the effectiveness of an online homework system to supplement instruction. However, (Chow, 2014) and Sosa et al. (2011) found that the students using the computer-based homework instruction are just as successful as those using the traditional method of homework instruction. Contrary, (Demirci, 2010) and Bonham, Beichner, and Deardorff (2003) reported that students who used the WebAssign system experienced no significant differences in learning gains comparing to students using the traditional hand-written homework. These findings implies that more rigorous studies, perhaps those control the experience of the instructors and students’ prior knowledge, are needed. The results also indicated that students’ attitudes toward use of Socrative as a homework assignment platform or toward preparing for exam through Socrative was moderate. Similarly, (Hadiri, n.d.) indicated slight positive changes toward Socrative when it was used for formative feedback. Other studies (Altun, 2008; Leong, 2014) concerning students attitudes toward use of Socrative was also positive. Some studies (Dervan, 2014; Godzicki, Godzicki, Krofel & Michaels, 2013); Heaslip et al., 2014; Manuguerra & Petocz, 2011) have indicated the increase in the students’ engagement when they use technology in or outside classroom. The results gathered from the open ended questions of study correspondingly specified that students became more engaged with their exams when Socrative was used as an online homework platforms.


319


The students in the sections utilizing the Socrative to complete the homework assignments are assumed to have increased students’ time on task in general. This most likely have positively impacted their ability to understand the material and influenced their performance on the final exam as well. In other words, the differences in students’ success can be attributed to the support and feedback that the software provides coupled with an increase in the time spent doing homework. Moreover, despite the fact that students in the treatment sections did significantly better on their final exams, there is still a group of students who scored incredibly low on their final exam. Still, the high correlation between students’ final grades and their amount of use of Socrative for preparation, indicate the general effectiveness of Socrative software. Obviously, doing homework sets with Socrative, the environment provides a sense of distance and protection from the audience not available in the classroom. Incorporating the Socrative in the exam preparation enables students to prepare for the exam with relish, regularly prepare for the exam and prepare more for the exam. Moreover, it provides opinions to questions from the instructor, receive feedback from the instructor while outside the classroom, and also the instructor assess their understanding of the course materials from distance. These results are valid for the statistics course, however, we envision that this would be the same if the Socrative be utilized in other, similar courses, as well. First the questionnaire was applied than students’ final scores were declared. This was done because of the probability of students being affected by their score on the final exam. There were two groups of items on the questionnaire; those assessing the Socrative and those assessing the usage of Socrative a homework sharing platform. The results indicated that attitude word Socrative (4.00) was more positive than using it for homework platform (3.71). The meaningfulness of the difference was searched by conducting independent sample t test. It was found that the difference was statistically significant; t(145)=12, p= .00. This finding implies that students have enjoyed with Socrative more than using it as a homework sharing platform. Which further implies that Socrative should be used for other purposes such as engagement and motivation. The students’ scoring the least on the eighth item might have been affected by their belief that the Statistics-I course was a difficult course. As a conclusion, the results revealed that preparation for exam using Socrative leaded to an increased exam grades. Further, results showed that the students are satisfied with this preparation style likely because the immediate feedback given helps to improve student understanding of the concepts. This study indicated that along with Socrative’s usual usage it can also be used to increase students’ performances on exams by sharing homework questions through Socrative. Based on the feedback received from students it is recommended that Socrative to be applied frequently in higher education institutions and in other subjects as well. Many students struggle to keep up with the courses possibly because of do not knowing how to prepare for exams and needs motivation. Instructors’ homework sharing on the web may be a solution to this problem. Further studies are necessary to generalize that the use of Socrative as homework platform will have the same learning results. References Altun, E. (2008). 6(Th), 7(Th) and 8(Th) Graders’ Attitudes Towards Online Homework Assignment Sites. Turkish Online Journal of Educational Technology, 7(4), 5–18. Awedh, M., Mueen, A., Zafar, B., & Manzoor, U. (2014). Using Socrative and Smartphones for the support of collaborative learning, 3(4), 17–24. Babaali, P., & Gonzalez, L. (2015). A quantitative analysis of the relationship between an online homework system and student achievement in pre-calculus. International Journal of Mathematical Education in Science and Technology, 46(5), 687–699. http://doi.org/10.1080/0020739X.2014.997318 Chow, A. F. (2014). Online homework impact in undergraduate mathematics and business statistics courses. Educational Studies, 5698(December), 1–5. http://doi.org/10.1080/03055698.2014.961902 Coca, D. M., & Slisko, J. (2013). Software Socrative and smartphones as tools for implementation of basic processes of active physics learning in classroom: An initial feasibility study with prospective teachers. European Journal of Physics Education, 4(2), 17–24. Demirci, N. (2010). The effect of web-based homework on university students’ physics achievements. Turkish Online Journal of Educational Technology, 9(4), 156–161. Dervan, P. (2014). Increasing in-class student engagement using Socrative (an online Student Response System). The All Ireland Journal of Teaching & Learning in Higher, 6(2), 1977. Hadiri, Y. (n.d.). Click it to check it : An instructional design module to assist university faculty in using Socrative as a smart student response system for student assessment . Heaslip, G., Donovan, P., & Cullen, J. G. (2014). Student response systems and learner engagement in large classes. Active Learning in Higher Education, 15(1), 11–24. http://doi.org/10.1177/1469787413514648


320


Krause, P. S. J. (2013). Just-in-Time-Teaching with Interactive Frequent Formative Feedback ( JiT- TIFFF or JTF ) for Cyber Learning in Core Materials Courses. Leong, K. E. (2014). College Students Attitude and Mathematics Achievement Using Web Based Homework. EURASIA Journal of Mathematics, Science & Technology Education, 10(6), 609–615. http://doi.org/10.12973/eurasia.2014.1220a Liu, D. Y. T., & Taylor, C. E. (2012). Engaging Students in Large Lectures of Introductory Biology and Molecular Biology Service Courses Using Student Response Systems, 154–162. Ongun, E., Altas, D., & Demirag, A. (2011). A STUDY OF 8(th) GRADERS’ PERCEPTIONS OF SOCIOCULTURAL PERSPECTIVE OF CREATIVITY BY USING INFORMATION TECHNOLOGY TOOLS IN REALISATION OF HOMEWORK GOALS. Turkish Online Journal of Educational Technology, 10(3), 21–26. Pallant, J. (2013). SPSS survival manual. McGraw-Hill Education (UK). Piatek, R. (2014). Student Response System : Student Activation towards Better Learning in Large Classes A Practical Guide TLHE Final Project, 1–16. Richards-Babb, M., Curtis, R., Georgieva, Z., & Penn, J. H. (2015). Student Perceptions of Online Homework Use for Formative Assessment of Learning in Organic Chemistry. Journal of Chemical Education, 92(11), 1813-1819. Socrative by MasteryConnect (2015, May 26). Retrieved from http://garden.socrative.com/ Sosa, G. W., D. E. Berger, A. T. Saw, and J. C. Mary. 2011. “Effectiveness of Computer assisted Instruction in Statistics: A Meta-analysis.” Review of Educational Research 81:97–128.


321