The Assessment of the Educational Quality in E ... - Science Direct

Procedia - Social and Behavioral Sciences 29 (2011) 1924 – 1931

International Conference on Education and Educational Psychology (ICEEPSY 2011)

The Assessment of the Educational Quality in E-citizenship Group of Tehran Technical Complex and Devising Optimization Strategies Shahroz Farjad ,NaderBarzegar,Nahid Hosseiny Islamic Azad University, Islamshahr Branch, Educational Administration Department (Tehran, IRAN, ISLAMIC REPUBLIC OF) Islamic Azad University, Islamshahr Branch, Educational Administration Department (Tehran, IRAN, ISLAMIC REPUBLIC OF) M.A. in Educational Technology (Tehran, IRAN, ISLAMIC REPUBLIC OF)

Abstract

The Quality of Educational services has been a matter of major concern to the educators. The education quality indicator is used as an assessment measure all over the world in this respect. The present paper focuses on the educational quality indices of E-citizenship Group of Tehran Technical Complex in form of the input and output of the education system. The study aims at pinpointing both strong and weak points of the above named educational body in order to facilitate the process of decision-making in such like institutions. : The study is an applied one and the data collection method has been descriptive. The statistical population consisted of students, teachers, graduates and Head of the selected centre. The data was collected through the application of sampling, questionnaires, interviews, and checklists. The analysis is done considering 10 factors, 43 standard measures and 84 indicators. The input index (=3.36) is acceptable, process index (=3.29) is inacceptable level, while output index (=2.32) falls below the standard level. The study result showed that the educational quality in the subject centre needed to be improved through implementation of optimizing measures and practical strategies.

©©2011 accessand/or under CC BY-NC-ND license. 2011Published Publishedby byElsevier ElsevierLtd. Ltd.Open Selection peer-review under responsibility of Dr. Zafer Bekirogullari Selection and/or peer-review under responsibility of Dr Zafer Bekirogullari. of Cognitive – Counselling, Research & Conference Services C-crcs. Keywords: Assessment; Educational quality; quality index; E-citizenship Group

1877-0428 © 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and/or peer-review under responsibility of Dr Zafer Bekirogullari. doi:10.1016/j.sbspro.2011.11.442

Shahroz Farjad / Procedia - Social and Behavioral Sciences 29 (2011) 1924 – 1931

1925

1. Introduction Major global changes make higher education systems in all over the world able to meet local needs and global issues and consider the rapid changes in global politics and demonstrate traditional educational programs, quality and effectiveness more clearly (El-khawas, 2006; Reported by Havas beigi, Keramati and Ahmadi,2011,3723). Moreover, globalization has been strongly influenced by the higher education and new challenges for control and management has emerged (Prince & Stella, 2007). Nowadays E-Citizen courses has been designed to help candidates get the most out of the internet by explaining how it works and showing that it can be used for a range of purpose. The public and private educational centers and institution have organized need for implementing new policies that empowers citizens by providing them with the chance of acquiring new skills, which are necessary for training them in E- citizens who are able to cope with the requirement of the modern life. These New skills include dealing with government departments, finding information, buying products and communicating online with family and friends. One completing of the program candidates will have acquired the skill to be able to fully participate in the E-society, use information resources in the difficult areas as well as able to use E- government services and other commercial services(e.g. banking) being to fully participate as one E- citizen. The Iranian educators have also realized that providing courses of this nature must become one of their priorities; among the centers that offered courses for training e- citizens, one can name Tehran Technical Complex. The present study aims to analyze these courses systematically through focusing on course input, process and output to facilitate the process of decision-making. The concept of educational quality is one of the major concerns of the every educational systems and educators. The on- going improvement of the educational quality is only made possible through educational evaluation (Patton, 1997). The subject of the study makes it necessary to define a number of terms at the outsets. The term quality has defined by various notions such as: superiority, zero mistake, serving the goals, modification, monetary value, and finally as outcome of important.(Campell,Rozsnayi,2002) The subject of educational quality has been the focal point of many researchers in this area (El-Khawas,1998, Birnbaum and Deshotels,1999,Campell and Rozsnyani,2002 ϭHarvey and Green,1993).

2. Research Review and literature Experience has proven that universities can provide best services to the community if they have concerns of continuous improvement in the quality of their services. Hence(2003) claim that evaluation is one of the strongest tools for strategic development in higher education environment. Evaluation of various courses of higher education is a necessity today, and also is the way to improve and increase quality of educational courses. (Yarmohammadian, 2004; Weber, 2003; Reported by Yarmohammadian, Mozaffary and Esfahani,2011, p: 2917). The term" Evaluation" refers to determining the real value of object, phenomena and events (Bisang and Zimmermann,2006). It can also be defined on the systematic appraisal of operation, outcomes, and consequences. The evaluation of an educational system includes the process of collection and analysis of data on the basis of

1926


certain measures to determine the feedback of the educational system and devising policies in order to improve the educational system. (Weiss,1984) Rossi and Freeman (1993) define evaluation as the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of programs. The role of evaluation is to provide answers to questions about a program that will be useful and will actually be used. Borrowing from Striven’s (1967) definition of Evaluation, Evaluation is different from research in terms of its unique purpose to judge the worth, merit, or quality of an evaluate, as well as in terms of the context-specific uses of the knowledge produced to make these judgments. (Reported by Marvin , Alkin and Sandy, 2003:p3). Evaluation is integrated part of Evaluation design Model and as a tool to help determine the effectiveness of educational interventionsEseryel,2002) It is also stated that evaluation of the educational system aims at providing of the evidence of the service offered (Chinapan&Miron,1990; Calder, 1995), the evidence is used to determine whatever the educational system have been successful.(Hong and Boden,2003) The evaluation of the system, as specified above, involves a systematic approach to the complex process of collecting data, observation, analysis, and value judgment. The main focus of system evaluation is to measure the value of the program on the basis of its merits.( Dykeman, MacIntosh, Seaman, & Davidson, 2003; Scriven, 1991) The evaluation of the educational program provides a better understanding of its feature and safeguards their participation in the program.(Stufflebeam,2000b) The following reasons make the evaluation the educational system an important process: -

The determination the strengths and weakness of the program

-

The determination of the priorities of developments plans

-

The determination of the areas that work well within system and the ones need to be improvement.

According to some researchers the four major objective of evaluation are: improvement, informing, stability and support. Any kind of evaluation requires a pre-determined set of standards. These quantitative standard help researchers to qualify then attempts within difficult area and make decisions only often considering all the strengths and weakness of the system.( Cace,2004) But there is no consensus on the definition of quality in higher education (Sun,2002; Yarmohammadian&Haeri, 2003; Yarmohammadian ,Bahrami&ForoughiAbari, 2008). Through evaluation, managers can get valuable information about effectiveness of programs; failures, strengths and weaknesses and compliance with setting quality standards and put the decision makers and educational planners in a better position to adopt the necessary measures to improve the methods to achieve goals and increase efficiency (ForoughiAbari, Yarmohammadian&Toroqi, 2004; Yarmohammadian et al, 2009). The Evaluation finding may assist such persons to make go/no go decisions about specific program modifications or, about initiation or continuation of entire program.(Rossi, Lipsey and Freeman,2004) Program evaluation is a continual and systematic process of assessing the value or potential value of Extension programs to guide decision-making for the program’s future.( Suvedi,Heinze and Ruonavaara,1999.p:2 ) The evaluation of an educational system can be defined as a systematic procedure of Collection and interpretation of


1927

data, concerning particular components of the system According to specific criteria and aiming at the feedback of these activities and efforts with useful information (Chinapah, & Miron, 1990; Calder, 1995, Reported by Pierrakeas, Xenos and s Pintelas,2003.p:335). In other words, Evaluation is a systematic process to determine the value and significance of a phenomenon (Phillips,1991). Evaluation is connected for a variety of practical Reasons: A) To aid in decisions concerning whether programs should be continued B) Improved, Expanded, or curtailed; to assess the utility of new programs and initiatives C) To increase the effectiveness of program management and administration; and to satisfy the accountability requirement of program sponsors.( Rossi, Lipsey and Freeman,2004) Also Program evaluation is critical for several reasons: It provides systematic ways to assess what needs improvement or what needs changing and it provides ways to validate practices, whether to internal or external audiences (Mishra, 2007; Reported by Travers and Evans,2011.p:123). Hoffman, Travers, Evans, and Treadwell (2009) studied 34 prior learning assessment programs across higher education institutions in the United States and Canada and determined five critical factors impacting program structures. These critical factors are 1) institutional philosophy statements and policies supporting prior learning assessment practices; 2) institutional support, including financial, administrative and faculty buy-in; 3) prior learning assessment program parameters that set the structures for how credit is assessed and applied; 4) faculty evaluator and content expert professional development; and 5) program feedback and evaluation processes. 3. The problem statement In the modern word, after the information boom, globalization and the need of training E- citizen who are familiar with the basic concepts of ICT and can surf the Net and retrieve the information needed, the application of ICT in the daily routine of life has become a crucial concern. It is to this goal that Tehran Technical Complex has designed and offered courses under title of " E- citizen Training" since 2005. The evaluation of the course content can be utilized in the following areas: - The evaluation of current course quality against the optimal situation, - The upgrade of the course quality, - The optimization of the course content, - Decision – making and devising the right polices, - The determination of strengths and weakness of the course.

4. Research Objective General objective: The evaluation and measuring the quality of educational system of E- citizen training course in Tehran Technical Complex.

1928


Specific Objectives: x Evaluation of course input Indicators such as (Teachers, course objectives, students, Equipment and Facilities). x Evaluation of course process Indicators such as (Teaching Method, Learning outcomes, management and organizational structure, course program). x Evaluation of course output Indicators such as (graduate's skills). x Suggestion practical guidelines to optimize the process of decisions- making to improve course quality and implement new policies. 5. Research Questions

1) What is the quality of the E- citizen course Tehran Technical Complex in regard to Input, Process and output? 2) How can the course input, Process and output be improved through modifications? 6. Research Methodology According to the nature of research, current research is “descriptive”, because it tries to recognize and describe the nature of Educational quality from the aspects of input, Process and output In Tehran Technical Complex. Research population The research population of the study consisted of the students, instructors and graduates of course as well as the head of department offering the E- citizen training course in Tehran Technical Complex. Sampling and sample volume To evaluate the course quality used to non probability Method, namely consensus (competing counting) and available sampling; The subjects consisted of 1 department head, 120 graduates, 30 teachers (instructors) and 50 students. Data collection, Analysis and conclusion: The means of data collections were questionnaires, checklist, and interview form.

Data Analysis: The collection data was analysis of data collected through questionnaires was done by applying Likert,s five point scale; points between 0 to 1.66 was assessed as unacceptable, between 1.67 to 3.33 fairly acceptable, and 3.34 to 5 as acceptable. 7.Finding of Research

1929


Question1. What is the quality of the E- citizen course in Tehran Technical Complex in regard to input, process, and output? Table 1:gives an account of the relationship between each factor and measures, indicators, as points and final judgment. factors

Indicators number

scores

Judgment

Instructors

15

3.54

acceptable

Course goals

8

3.58

acceptable

students

4

3.38

acceptable

Curriculum

17

3.16

fairly acceptable

Equipments

10

3.18

fairly acceptable

Teaching &Learning Method

7

3.44

acceptable

management and organizational structure Graduate

10

3.32

fairly acceptable

7

2.35

acceptable

Total

78

3.24

fairly acceptable

Table 2: gives the number of factors measures and indicators as well as the points and judgment on the input, process and output of the course. The overall points of the course (2.99) is fairly acceptable

Question2. How can the course input, process, and output be improved through modifications? The Research Result show that these factors can lead to increase of educational quality in E- Citizen Courses in Tehran Technical Complex: Item

Factors number

Indicators

scores

Judgment

number Input

5

45

3.36

acceptable

Process

3

20

3.29

fairly acceptable

Output

2

13

2.32

fairly acceptable

Total

10

78

2.99

fairly acceptable

-

A comparativ e study of the course content and objective, reviewing the

1930


-

international standards, upgrading and on- going modification of the course content. Educating competent instructors of E- learning and E- citizenship course in the higher Education centers and universities. Establishing the required infrastructure (software and hardware), proving broad band and fast internet services. In cooperative virtual and face to face instruction on E- citizenship. Making people (general public) willing to know more about virtual services and E- citizenship. Using mass media to inform and educate general public to remove Technophobia. Using state-of- art Teaching Methodology, Improving standard learning through simulation another simulation and other Techniques.

Conclusion: Nowadays, Educational Quality issue emphasized by many of educational institutions in the all of the world. Emphasize to Educational courses quality with systematic Approach include Input, Process and Output is very important. Also, Defining Educational Assessment Indicators can lead to increase of Educational Quality in institutions. The research result indicated that Although E-Citizenship course in Tehran Technical Complex has a fairly acceptable scale in the Educational Quality but it is far from ideal situation. Thus, present research can help the managers of the Course to better decision making to increase the educational quality. References: Birnbaum, R. and Deshotels, J. (1999), “Has the academy adopted TQM?”, Planning for Higher Education, Vol. 28 No. 1, pp. 29-37. Bisang.kure.,Zimmermann.,willi.(2006). key concepts and methods of programme evaluation and conclusions from foresty practice in switzerland. Forest policy and economices (2006).502-511. Boden,Mark, Hong,Heung.(2003). R&D Programme Evaluation: Theory and Practice. Cace, C., (2004). Management of social insurance. Tendencies and evolution.Expert Publishing House, Bucharest. Calder,J.(1995). Program Evaluation and quality: a comprehensive guide to setting up an evaluation system Institute of Educational Technology, open university. London: Kogan page. Campell, C. and Rozsnyani, C. (2002), Quality Assurance and the Development of Course Programs, Papers on Higher Education, Unesco-CEPES, Bucharest. Chinapah,U.,&Miron,G.(1990). Evaluation educational programs and projects: holistic and practical consideration. Socio- economic studies,15.Paris:UNESCO. Christina A. Christie and Marvin C. Alkin(1999). Future Reflection on miutilization. Studies in Educational Evaluation 25 (1999) l-10. Christos P, Xenos ,M and Pintelas, P (2003). Evaluation and Improving Educational. Dykeman, M., MacIntosh, J., Seaman, P., & Davidson, P. (2003). Development of a program logic model to measure the process and outcomes of a nurse-managedcommunity health clinic. Journal of Professional Nursing, 19(4), 197–203. El-Khawas, E. (1998), Quality Assessment in Higher Education: Recent Progress; Challenges Behind, The World Bank, Washington, DC. Eseryel,Deniz.. Approaches to Evaluation of Training: Theory&Practice. Educational Technology&society5 (2)2002. Foroughi Abari, A. A.,Yarmohammadian, M. H.& Toroqi, J. (2004). Effectiveness in Higher Education, Encyclopedia of Higher Education, Ministry of Science, Research, and Technology, Tehran, Iran. Harvey, L. and Green, D. (1993), “Assessing quality in higher education: a transbinary research project”, Assessment and Evaluation in Higher Education, Vol. 18 No. 2, pp. 143-9.


1931

Havas beigi.F., Keramati.M..and ., Ahmadi.A.(2011) The quality curriculum evaluation in postgraduate studies of educational management and planning in the public university of Tehran city Academic World Education&Research Center, 3rd World Conference on Educational Sciences – 2011. Hoffman, T., Travers, N. L., Evans, M., & Treadwell, A. (2009, September). Researching critical factors impacting PLA programs: A multi-institutional study on best practices. CAEL Forum and News. Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, NJ: Sage. Stufflebeam, D. L. (2000b). The CIPP model for evaluation. In D. L. Stufflebeam, G. F. Sun, C. B. (2002). Quality in higher education: Policies and practices. A Hong Kong perspective, introduction and research approach. Joural or educational management. Vol.17. No 3. Suvedi,M. Heinze .K and Ruonavaara.D(1999). How to Conduct Evaluation of Extension Programs. ANRECS Center for Evaluative Studies. December 1999. Marvin C. Alkin and Sandy M. Taut(2003). Unbanded Evaluation use. Studies in Educational Evaluation 29 , l12. Patton,M.,1997.,utilization-focused Evaluation: The New Century Text,3rd.Edition.Sage, Thousand Oaks. Philips, J .(1991) . Handbook OF Training evolution and measurement method (2nd ed), Houston : Gulf Publishing company. Prince, M., & Stella, A. (2007). External quality assurance in higher education: making choices: paris : UNESCO: international institute for Educational planning. Roosi,P.Freeman.H.(1993).Evaluation: A systematic Approach. Sage Publication.5th Edition. Rossi.H.Peter, Lipsey.W.Mark, Freeman.E.Howard(2004). Evaluation- A Systematic Approach-7th edition.Sage Publication, Inc. Stuffelbeam, D.L.(2001) , Evolutions Models .Scan Francisco : Jossey – Bass. Travers. L. Nan, Evans. T.Marnie.(2011). Evaluating Prior Learning Assessment Program: A Suggested Framework. International Review of Research open and distance; Learning. Vol.12.1. January. Weiss,C.H.,1998. Evaluation.Evaluation, second edition. Prentice Hall,Upper Saddle River,NJ. Yarmohammadian, M. H., Haeri, M. R. (2003). Quality in Higher Education via accreditation; first international conference on management, Banglore, India, 28 dec.-1 Jan. Yarmohammadian, M. H. (2004). Quality in Higher Education, Encyclopedia of Higher Education, Ministry of Science, Research, and Technology, Tehran, Iran. Yarmohammadian, M. H., Bahrami, S.& Foroughi Abari, A. A. (2008). Educational Administration & Planning, Health Management&Economic Research Center, Isfahan University of Medical Sciences, Isfahan (Book in Persian). Yarmohammadian, M. H., Haghshenas, A., Raeisi, A.& Asgari, H. (2009). New Integrated Model for Strategic Planning in Higher Education, 6th International Conference on Management, Greater Noida, India.Dec.28,2008-Jan. 1,2009. Yarmohammadian, Mohammad Hossein; Mozaffary, Mina; Esfahani,sekineh Saghaeiannejad(2011). Evaluating Quality of Education in Universities Based on Academic Quality Improvement Program ( AQIP) Model.Academic World Education&Research Center, 3rd World Conference on Educational Sciences – 2011.