NEW TRENDS IN MONITORING, TESTING AND GRADING OF STUDENTS OVER THE INTERNET Frank Schindler Department of Applied Computer Science and Engineering Faculty of Electrical Engineering and Informatics Slovak Technical University, Bratislava, Slovakia E-mail:
[email protected] Abstract At the present time, learning management systems (LMS) represent the most adaptable and flexible method of contemporary electronic learning in which a collection of interactive programs are exclusively designed and used to develop, organize and offer online courses over the Internet. They make accessible a variety of novel online individual and group activities to prop up and complement student's self-study. Nonetheless, in order to take a full advantage of them an innovative way of testing seems to be not only necessary but also more beneficial. It is pretty evident that monitoring and testing of such virtual students must be entirely different from what we all are used to in the face-to-face lectures and in addition to that it ought to be more or less ongoing process. Those, who have an experience with e-learning, probably will agree with me, that monitoring, testing and grading of online students is quite a challenging job. To make it manageable and bearable for the online teacher, each LMS system should offer some additional built-in features that make monitoring, testing and grading of students more digestible. In this article I would like to share with you a few of my experiences that I gained during the period of last six years of the time when I was teaching online web-based undergraduate Computer and Information Science classes via WebTycho for the University of Maryland University College1 (UMUC).
1. Introduction Modern information and communication technologies caused an upheaval in existing educational world, because they allow to feel information through more senses. In view of that, when used correctly they can achieve higher effect in learning. Let me remind you, that in our everyday life we are constantly swamped with a massive 1
University of Maryland University College, European Division, Im Bosseldorn 30, D-69126 Heidelberg, Germany
amount of new data demanding gradually rising requirements on the level and quality of our education. Consequently, there is a need to apply new forms, methods and also study support instruments that are capable to gather, process and picture a wide-ranging array of information. At the moment, learning management systems (LMS) stand for the most flexible technique of present-day learning. They involve a group of software program integrated together and intended to develop, organize and direct online lectures under one specific virtual environment. The main aim of this paper is to talk about different means of monitoring, testing and evaluation of online students in a distance learning course offered over the Internet. As I metioned above, learning management systems are a commonly established tool to support such virtual study. Electronic and multimedia study materials are often implanted into them to provide superior simulation, visualisation and/or understanding of studied concepts. It is well known that acquisition of new knowledge is much easier when we already know similar things and thus we have created in our memory an internal structure that allows grasping this kind of new information. On the other hand, if we try to learn an entirely new information we have to generate from scratch also a totally new arrangement linked with it and thus the learning process gets not only much more difficult, but also much more complicated (see [2]). This could be facilitated when we first concentrate on a model creation implied by know-how obtained by solving analogous tasks. We have to bear in mind, that no matter of how attractive, ingenious and simple presentation of knowledge is it does not explicitly support any creation of such internal structures. Therefore, all this implies, that besides fancy presentation of concepts via electronic study materials in an LMS, a vital part of every such online learning must include online testing and evaluation of students. Indisputably, that may be quite a tricky matter, since implementation of it in an LMS system harbors a lot of possible pitfalls. The cause for that is that most of online tests until now entail only a limited type of
questions, e.g. true/false, fill-in or multiple-choice questions. Such online tests often lead to memorization of the test questions along with their respective answers (see [3]). That could be counterproductive, since it does not openly encourage imaginative thinking and also it does not test student's overall knowledge from the subject matter being tested. Advantages of online testing are quite obvious. Simple tests of that kind can be marked (graded) mechanically without human intervention almost continuously. Since it uses the technique to fill-in or point-and-click or to pick-a-correct-answer, machines will evaluate them more or less instantly and thus the student gets his/her score right away after the test is over. For that reason we refer to this method of testing as self-testing. Preparation of such tests might be quite a demanding job, but unquestionably it can be done (see [1]). Therefore it would make a perfect sense to design a computer program generating a variety of such online tests created by pressing a key on the keyboard after the tested student replies to a few simple questions regarding the outline, content and complexity of the expected exam. Whole series of model self-tests could be utilized anytime in the self-study. As example of a new trend in testing of students I would mention one paper textbook (see [1]) that comes along with a CD-ROM containing not only a complete instructor's guide along with transparency masters for face-to-face lectures, but also a complete set of sample solutions to numerous chapter review questions, and an interactive program generating a diversity of more than thousand of trial test questions of varying difficulty chosen from various chapters that may be created by pressing a key on the keyboard after the user answers a few easy questions concerning the shape and difficulty of the proposed exam. No joking, such a textbook would be of an immense help in any kind of successful virtual learning course. It is exactly what we all need at this juncture in order to test online students. Also an open question remains if this is what we would like for students to get access to. Though it seems to be fantastic that students are able to test their knowledge by themselves. Of course, too much of weird testing could hurt new students because instead of studying they might go over the top with testing. On the other hand extensive self-testing of acquired knowledge seems to be just one of the features that could be implemented in an electronic manner as a part of the virtual classroom in an LMS system. Online study should never shrink to self-study. Therefore I advocate usage of virtual learning communities for the sake of testing and evaluation of students. They should be easily set up via online discussion forums accessed over the Internet. Via them online assignments and tests similar to the classical ones known from face-to-face
classes might be administered. Any kind of question may be incorporated into an MS Word document, which in turn can be uploaded into a corresponding online study group that is ready to take the online exam at the prearranged time. A discussion forum allows formation of an asynchronous dialog to share knowledge in a group, in which an additional information needs to be created. At the very beginning of the dialog, the teacher ought to make a short introduction into the topic and should pose a question (task) to be solved in the forum. If the participation of students is not obligatory, only 2-3 students will do the task. Therefore it is not of a disadvantage to request all students to solve the task. In addition to that the teacher ought to add to his task some subtasks that are given to a specific student depending on his actual or partial solution. Teacher should watch closely over the actions in discussion forums and guide them if necessary by making a short summary or posting new sub questions. This sort of activities in the virtual classroom are useful only provided there are not too many of them. You should take that into account, otherwise students are going to skip them due to shortage of time.
Fig. 1 Summary of All Conference Participation for One Specific Student in WebTycho
Fig. 2 Specific Conference Responses of One Particular Student in WebTycho
Fig. 4 Online Assignment #1 as Attachment Submitted into Student’s Assignment Folder in WebTycho
Fig. 3 Grades for Online Assignment #1 Entered into Gradebook in WebTycho – Teacher’s View
Fig. 5 Complete Portfolio of All Work Done by One Specific Student in WebTycho
deal in the evaluation process. See figures 1-6. Features such as “portfolio of all work” done by a specific student are extremely advantageous. Without them online grading would be a nightmare. Most of this I have been successfully using in my distance education courses that I teach since 1999 for UMUC over the Internet via WebTycho.
3. References [1] S. D. Burd, Systems Architecture, Fourth Edition, Thompson Course Technology, 2003, pp. 1-638, ISBN 0-61915978-2. [2] M. Huba, K. Žáková, P. Bisák, Development of On-line and Web-based Training, ŠEVT a.s., 2005, ISBN 80-227-2237-5. [3] P. ČIČÁK, I. GRELLNES, M. KOTOČOVÁ, P. MESJAR, Contribution to the Evaluation Methodology, Proceedings of Conference „ICETA 04“, Košice, 2004, pp. 515-518, ISBN 8089066-85-2. [4] D. Hamilton, et al., Beyond the Numbers Game: A Reader in the Educational Evaluation, Macmillan, London 1977.
Fig. 6 Overview of Online Course Grading via Gradebook in WebTycho
[5] P. HOROVČÁK, B. STEHLÍKOVÁ, Electronic Testing Evaluation of Results, Proceedings of Conference „ICETA 04“, Košice, 2004, pp. 307-312, ISBN 80-89066-85-2.
2. Conclusion
[6] P. HOROVČÁK, B. STEHLÍKOVÁ, Electronic Test - The New Form of Communication Between Students and Teacher, Proceedings of Conference „ICETA 04“, Košice, 2004, pp. 313318, ISBN 80-89066-85-2.
Each testing and evaluation of online students should be closely related to such concepts as "reflection" and "feedback" and it ought to express an effort to understand underlying processes in the virtual course, in order to be able to form an opinion and/or imply necessary study decisions based on it. Evaluation should not only deal only with intended activities in the course, but also it ought to reflect authentic happening in the course. To make the online education meaningfull and effective, it is very important that students know how to efficiently test and evaluate their knowledge online in the nick of time. They should understand, accept and agree with their online evaluation. To make that some sort of a verbal feedback from the teacher is necessary. Not all students are able to correctly assess their actual knowledge. In an ideal case teacher's evaluation of the student should coincide with his/her self-evaluation. Therefore in every distance education class clearly defined evaluation criteria must be specified at the very beginning of each online class. Moreover properly made evaluation should motivate and guide students in their self-study. In some instances verbal evaluation of the student's work might be needed, e.g. in particular at the evaluation of student's work completed inside of discussion forums. Exhaustive grading of all online work of students is not only time consuming, but also needless and therefore it should be avoided. An LMS system could help the teacher a great
[7] M. Morris, M., R. Twitchen, Evaluating Flexible Learning: A User's Guide, National Foundation for Educational Research, Slough 1990. [8] F. Schindler, WebTycho: A Tool for Distance Education over the Internet, Proceedings of International Conference "E-Learn 2004", University of Žilina, 2004, pp. 231-235, ISBN 80-8070-190-3. [9] F. Schindler, Evaluation in Online Distance Education, Proceedings of Colloquium „DIVAI 2004“, Nitra, 2004, 4 pages, ISBN 80-8050-691-4. [10] R. SČESNÝ, P. KRIŠTOF, Possibilities and Limitations of the Creation of Tests for E-Learning, Proceedings of Conference „ICETA 04“, Košice, 2004, pp. 377-384, ISBN 80-89066-85-2. [11] P. SOUKUP, E-Learning and Checking of Study Effectiveness by Testing, Proceedings of Conference „ICETA 04“, Košice, 2004, pp. 371-376, ISBN 80-89066-85-2. [12] M. Thorpe, Evaluating Open and Distance Education, Longman, Harlow 1988.
