The former projects such as Primary Education for Quality Improvement ... Indicators of the inefficiency of education in Indonesia are the repetition rates, the.
Research Paper
The Quality Improvement of Primary Children Learning through a School-Based Programme in Indonesia
Laurens Kaluge Setiasih Harjanto Tjahjono
Universitas Surabaya East Java, Indonesia 2004
1
Table of Contents Table of Contents Abstract Acknowledgements 1. Background of the Quality Improvement Programme 2. The Sidoarjo School Improvement Programme 3. Methods of the Study 4. The Description of Data 5. Findings and Discussion 6. Conclusions and Recommendations References Appendices
2
Abstract This paper reports the evaluation results of a school-based quality improvement programme called SSIP in East Java, Indonesia. In terms of equity, the evaluation took into account variables such as gender, age, and family social economic status referred to parental education and occupation. The programme focused on Social Studies in Grade 3 and 4 primary schools. The main problem was: after controlling for the pupil, class, and school factors did the programme improve the quality of schools? There were 19 experimental and 15 control schools identified on the basis of local clusters included in the study. Thus the study followed used a quasi-experimental design and multilevel modelling techniques (using pupil, classroom and school as the levels) were used to analyse the data. Even though not all of the results were consistently evident across all analyses the following influences were generally identified. Prior attainment (for Grade 3 and 4), ability, gender, father education, parental occupation, self-concept, democratic attitudes, locus of control were significant variables at pupil level. Classroom management was important at the class level. Formalisation, teacher affiliation, and work pressure were important factors for the children’s progress at the school level. The experimental schools achieved better than the control schools. Before entering any factor, the variance explained at the class and school level ranged between 16.3 and 29.4% for Grade 3 and 27.3 to 46.2% for Grade 4. After including the significant factors in the analysis, the contribution of the final model ranged from 16.3 to 36.8 % for Grade 3 and 27.8 to 50.3% for Grade 4. The success of such an educational improvement programme is optimistic sign for the future.
3
Acknowledgement This is a report on the evaluation of a school-based programme in order to improve the quality of children learning. Since the initiation of the programme until finishing this study the researchers are grateful to many institutions taking part in different time, ways and places. Firstly, The Universitas Surabaya and FINED as sponsors for the programme. Secondly, the office of education such as District Kanwil and Local Kandep Diknas as superordinate of schools which positively support to the activities of the programme. Thirdly, schools, principals, teachers, and pupils as the main actors in the programme.
4
1. Background of the Quality Improvement Programme 1.1 Current problems of Education in Indonesia The Republic of Indonesia has achieved general education for almost all children in Indonesia. However there are several problems in education to be discussed for research and improvement projects. The important problems are the effectiveness and efficiency of education. With effectiveness is meant the effects of schools, teachers and education in general on the student achievement. The effectiveness is reflected in the outcomes as measured by standardised tests, number of students promoted to the next grade after taking into account the background of students, their prior knowledge and the socio-economic status (see Teddlie & Reynolds, 2000; Creemers, 1994a; Gray & Wilcox, 1995). In general the output of education has been lower than expected, especially in more disadvantaged areas. In this case, the great discrepancy between schools need to be considered. Some schools are doing quite well but there are also schools that perform quite poorly. The efficiency of education is negatively influenced by the amount of grade repetition, dropouts and there are indications that schools cannot attain their objectives in a given time-framework and that the cost of education increases with spending more time in attaining the goals for specific children (Scheerens, 1992; Thomas, 1992; World Bank, 1998; Unicef, 2000). The former projects such as Primary Education for Quality Improvement Project (PEQIP), and the Basic Education Project (BEP) tried to increase the effectiveness and the efficiency of education. As is known from other international and national project studies, the effectiveness of education is especially related to the instructional processes at classroom level and the organisation of the school. In this respect, the major problems of Indonesian education become clear. The problems concern the instructional processes such as objectives and content of instruction, the activities of students and the supportive conditions at the school level. The ordinary classroom in Indonesia is characterised by a low degree of activity of children. They are sitting in rows, not actively involved in the learning process. One of the reasons might be that the instructional process is not attractive, not well structured and doesn't contain the other characteristics which make instruction effective. In terms of equity, it is also not related to the differences between children because the instruction treats the classroom as a whole and is not adapted to the needs of specific children. In project for Active Learning through Professional Support (ALPS) this issue was partly aimed at increasing the involvement of children in learning activities (Tangyong, Wahyudi, Gardner, 5
& Hawes, 1989). Other characteristics, however, of effective instruction were somewhat disregarded in this project. One of the causes of poor learning achievement of students in Indonesia is the centralized curriculum, which is ineffective. The central ruling about the curriculum cause limited teacher creativity. A second problem is the inadequate assessment of student achievement. So far, efforts to improve student achievement have focused almost exclusively on inputs, such as teacher training, curriculum revision and provision of textbooks. But students, teachers and principals do not work hard together with the intention to achieve the objects of education. A third problem is the lack of teaching and learning materials. Several studies pointed out that there is a bad distribution of the textbook and if the textbooks are available, many times students and teachers don’t know the proper use of the books. Another problem is the number and quality of the teachers. A number of studies indicate that teacher competency in Indonesia is low. Consequently, many Indonesian teachers working in primary schools have educational qualifications lower than formally required. The quality of education is not only related to the curriculum delivery and instructional technology but also the content of education and instruction itself. The objectives and content of Indonesian education is academically focussed. The academic subjects (mathematics, language and science) in the majority of schools are not related to daily life experiences of children nor can what is learned be practiced in daily life again. In the cognitive domain basic knowledge and skills are addressed, but there is not much attention for higher order skills, like analysing, synthesising and the evaluation of knowledge. Furthermore there is little attention for educational objectives in other domains (aesthetic, social, affective and moral). In daily life however children deal with more than cognitive knowledge and skills alone, they have to deal with social, cultural and moral situations as well. It can be expected from education that education at least provides a cognitive basis to deal with this kind of topics. One of the causes of poor learning achievement of pupils in Indonesia is the centralized curriculum, which is ineffective. The central ruling about the curriculum causes limited teacher creativity. Another problem is the inadequate assessment of pupil achievement. So far, efforts to improve pupil achievement have focused almost exclusively on inputs, such as teacher training, curriculum revision and provision of textbooks. But pupils, teachers and principals do not work hard together with the intention to achieve the objects of education. A third problem is the lack of teaching and learning materials. Several 6
studies pointed out that there is a bad distribution of the textbook and if the textbooks are available, many times pupils and teachers don’t know the proper use of the books. Another problem is the number and quality of the teachers. A number of studies indicate that teacher competency in Indonesia is low. Consequently, many Indonesian teachers working in primary schools have educational qualifications lower than formally required (Thomas, 1992; Supriyoko, 2002; Suparno, 2002). Indicators of the inefficiency of education in Indonesia are the repetition rates, the dropout rates and the non-completion rates. One of the causes of the inefficiency is the economic crisis and dropout rates in earlier grades. Because of the crisis many parents have low income and don’t have enough money for the education of their child. Dropout rates are especially high in the higher grades (Unicef, 2000). Students repeat because they cannot achieve the minimum standard of academic achievement, but repeating is also closely related to the child’s readiness. The attendance at pre-school or kindergarten has a positive impact on student’s performance. However, only a small proportion of children attend kindergarten or pre-school programs. Internal inefficiency also seems to be a problem (Unicef, 2000). Surakhmad (1999) prescribes a practical-technical approach to solve current problems in Indonesian education. For better education there should be a future oriented policy, a democratic foundation of education, a predominantly pioneering approach and a decentralized, community base management of education. The project of SSIP also uses other approaches, which set priorities in improving the curriculum and improving teacher competencies, in order to improve Indonesian primary education. 1.2 Former school improvement projects in Indonesia Several school effectiveness projects (Kaluge, 1999; Ministry of Education, 1998) are considered as the background for SSIP. A study by BP3K sponsored by the Indonesian Ministry of Education and Culture (Setijadi, Moegiadi, Wiradinata & Elley, 1978) concluded that the largest influence on student achievement were the indices of classroom climate. Another important conclusion made was that the students in city schools achieved better than the students of the rural schools. The results of this study were used as some starting considerations for innovation in Indonesian primary education, known as CBSA (student active learning), introduced by the ALPS project (Tangyong, Wahyudi, Gardner & Hawes, 1989). 7
Suryadi (1992) conducted another Indonesian study under the sponsorship of the Indonesian Ministry of Education and Culture and USAID to improve primary schools. He found that home and community variables affected variation more than the school quality variables. The study was held in three provinces over 5790 random sampled students. Quality and length of the instruction time were found out to be the most strongest policy manipulable predictors of student achievement. Positive factors to higher student achievement were reading books, peers discussion, teaching through reading, the use of teaching materials and class discussions. Lecturing by the teacher was a negative factor to higher student achievement. Some of these studies suffered from severe methodological problems, because they failed to considered important baseline variables (Kaluge, 1999). The Primary Education Quality Improvement Project (Ministry of Education, 1998) focused on the training of tutors and subject matter specialist, who then in their turn had to train teachers. Focus of the study was the professional development of teachers, the educational management, books, learning materials and evaluation and monitoring. PEQIP affected the quality of education in terms of student outcomes. However, differences between PEQIP schools and non-PEQIP schools were small. The implementation time was too short. The quality of instruction (the professional development of teachers) turned out to be the most important factor to higher student achievement. The PEQIP-model was not effective from an implementation point of view. Therefore, the authors made recommendations on several levels. The focus of the principals is too much focused on administrative tasks and to little on educative tasks. The training of teachers should focus on characteristics of effective teaching. The involvement of community and parents deserves further attention. Good textbooks that are actually available and used in classrooms, as well as teaching aids and learning materials, are important vehicles for educational improvement. The SSIP uses information of all these studies. The programme was meant for all demographic types of schools and would be conducted on a small number of schools. Teachers would be trained in order to make a better class climate and to improve the quality of instruction. Principals would be involved in order to let them focus on educational tasks, instead of administrative tasks. Group discussions would be held. Teaching materials would be used. Intake variables would be taken care of.
8
2. The Sidoarjo School Improvement Programme 2.1 The Programme Objectives The Sidoarjo School Improvement Programme (SSIP) used information of all those previous studies. The programme was meant for all demographic types of schools and was conducted on a small number of schools. Teachers were trained in order to make a better class climate and to improve the quality of instruction. Principals were involved in order to let them focus on educational tasks, instead of administrative tasks. Group discussions were held. Teaching materials were used. Intake variables were taken into account. Education in Indonesia is characterised by the fact that there are very strong central guidelines with respect to the objectives and content of education. The room for experimentation for individual schools is quite small. The only part that is more flexible and more open is the local content part. Because this programme was only a pilot project, it was decided to situate the project within this local content part in order to improve the quality of education for that part of the curriculum. This programme was aimed at improving quality in schools. There were four specified objectives: •
develop and implement objectives in cognitive, social and affective domains, especially the synergy between the three domains
•
develop up to date learning opportunities in the form of teaching-learning packages and implement those packages in educational practice
•
improve the quality of instruction within the programmes by improving the methodology and the procedures of instruction, stressing more active involvement of pupils, more adaptivity (related to specific needs of individual pupils) and improve instructional methodology of the teacher.
•
improve the conditions for classroom learning and teaching at the school level to support improvement establish school – University partnership. The knowledge basis for this programme is provided by research in different areas
and especially educational practice, which was evaluated. The specific characteristics of this programme was that it was not restricted to one kind of knowledge basis or a certain topic within the curriculum but it tried to combine different knowledge bases from different areas. The main areas were (i) school improvement (ii) curriculum development and (iii) teaching strategies. 9
2.2 School improvement. Of course the improvement has to reach the children and that goes through the classroom where teaching and learning take place. But in order to achieve classroom effectiveness, conditions at school level have to be fulfilled, such as support at the school through teaching materials and the supervision of teacher's teaching and the monitoring of teachers' and pupils' progress. It also includes that the support given by the experts of the teacher training institution and the universities. The support was provided in the school. Teachers were not be taken out of their school and out of their classes. The supervision of teachers and the guidance and monitoring was done through feedback on the actual classroom behaviour. This can be done by the principal but also by external facilitators. Schools have to develop an organisation within the school where innovation and improvement can take place and is supported. There are different strategies for school improvement based on studies by Stoll and Fink (1996), Hopkins (1990), Gray and Wilcox (1995), Gray, Reynolds, FitzGibbon and Jesson (1996). The main focus were the change of quality of teaching and learning at classroom level by creating an organisation planning, implementing and evaluating the process of teaching and learning in schools. Specific strategies which turned out to be effective for teacher improvement was used, such classroom consultancy where teachers immediately got feedback after the teaching about their teaching. 2.3 Curriculum development. It is well-known that curriculum on paper doesn't work and that the final proof of the curriculum is in the implementation of it. Curriculum Research resulted in some principles can guide the development of materials. Materials should be well structured, clearly focused and provide a lot of practical suggestions for activities, evaluation, feedback and adaptation to individual needs. The curriculum was developed to be up to date and reflect the environment where children live in. This was based on "old philosophies about education" (Dewey) and new theories about the construction of knowledge by pupils themselves and theories and experiences with real life mathematics and other real life educational programmes (the constructivist approach). 2.4 Teaching strategies. Educational research increased our knowledge about effective instruction, especially on strategies which teachers could use to start, maintain and evaluate learning of pupils. 10
Characteristics of effective instruction can be found in the curriculum materials (learning packages), the grouping of pupils in the classroom and - especially- the instructional activities of teachers (Creemers 1994a, 1997). Although mastery learning was a very effective way of grouping, in this programme we chose collaborative learning (Slavin, 1996) in order to achieve certain goals. To increase teaching effectiveness the model of direct instruction (Rosenshine, 1987) was applied with - if possible - some more constructivist approaches (Creemers, 1994b; Creemers & Reezigt, 1996). 2.5 Activities. The pilot project consisted of the following interrelate activities. Firstly, exchanging information and finalising the project plan. The activity information was done by experts from the IKIP/university (teacher training institution). The expert (supervisor) provided further information about the different components comprised the project. This was discussed because, finally, schools together with the experts had to agree on the project plan and the activities. Secondly, the development of curriculum materials (the teaching-learning packages). This was done by teams of teachers and principals within the gugus (school cluster), depending on the number of gugus and schools which participate. Teams of teachers and principals within the gugus (school cluster) had to develop about 6 packages, three for each Grade 3 and 4. The development of the packages was supervised by the experts of the universities. It was expected that different phases of construction were necessary: 1) draft, (2) presentation and discussions to other participants who have to work with it as well, (3) improvement based on critique, (4) pilot testing in different grade level and schools, (5) final form, and (6) production of materials. Thirdly, training the teachers and principals. These activity started immediately and continued throughout the whole project (in all stages). Fourthly, the implementation of the curriculum. In this activity the teachers carried out the package. This was supervised by the principal. In this way teachers got immediate feedback from the experts. Gradually the principal took over this task from the experts. Fifthly, evaluation. In the evaluation information was collected about the opinions of teachers, principals and students concerning the packages in order to improve the curriculum and finally the achievement of students in different areas. The information was analysed. Sixthly, reporting. The development of materials (phase I), was reported separately from the implementation of the materials in the second phase. The intention was also to publish a final report which addresses also the question of school/educational improvement in this project. It 11
gave recommendations for future school improvement projects (based on the results of students and teachers).
12
3. Methods of the Study 3.1 Objectives of the Study The purpose of this study was providing information on to what extent school had been improved by introducing the SSIP after taking into account the pupil, class, and school factors. In more specific, the objectives of the study were attempting to answer the following research questions: 1) what were the meaningful variables at pupil, class, and school levels that differentiate the children progress in learning the social studies? 2) did the experimental schools (those that implement the programme) achieve success and how big the variation accounted for the pupil and school or class levels? 3.2 Sample This programme is located in the Sidoarjo-district, south of the city of Surabaya, Indonesia. In this programme, it was chosen three sub-districts where in each has school clusters named gugus. The participating subdistricts were Sidoarjo (urban characteristic), Krian (sub-rural characteristic) and Krembung (rural characteristic). Each gugus was divided into experiment schools and control schools. Table 3.1 shows the number of schools per gugus. The number of control schools of Gugus Krian is low. Due to circumstances it was not possible to arrange more control schools. Table 3.1 Number of participating schools Sidoarjo Krian Krembung Total number of schools
Experiment Schools 7 6 6 19
Control Schools 5 3 7 15
The teachers in Grade 3 and 4 had an average age of 43 years. The teachers of experiment schools were in average one year younger than the control teachers. 81% of the teachers was female (88% at the experiment schools and 74% at the control schools). Both experiment and control school teachers have an average experience in teaching of 13 years. The average order of promotion rank between experiment and control schools was almost the
13
same. All differences between teachers of experiment and control schools were not significant. These 34 schools contained 1884 pupils in Grade 3 and 4. In this report a sample of 1547 pupils were used. During the testing some pupils were absent. Therefore in this section only results of pupils who were present at all the tests (Intelligence Test, Content Related Test in Cawu* 1 & 2, pupil background, Smiley Test) would be used. There was also an outliers test conducted, as well as a distribution test. This way, the number of the pupils was reduced to 1547. The number of these pupils per gugus per grade was shown in Table 3.2 and were used in all results in this report. Table 3.2 Number of participant pupils
Sidoarjo Krian Krembung Total number of pupil
Grade 3 Experiment Control Schools Schools 154 138 195 31 94 169 443
338
Grade 4 Experiment Control Schools Schools 137 100 165 37 114 213 416
350
Total
1547
The average age of the pupils in Grade 3 was at time of measuring 7 years and 5 months. The average age of the pupils in Grade 4 at the same time was 8 years and 7 months. There were more male than female pupils in the schools, except for the experiment schools of Gugus Krembung. The experiment schools have 52% male pupils; the control schools have 53% male pupils. Gugus Krembung was showing the most equal distribution between male and female pupils. Differences between experiment and control schools and between gugus were not significant according to a Chi-square calculation. The 34 schools were divided between 19 experiment and 15 control schools. Average age of the teachers is 44 years old. In experiment schools 28% of the teachers was male, in control schools this was 34%. The average order of promotion rank between experiment and control schools is almost the same. The teachers in the experiment schools have overall less experience in teaching than the teachers in the control schools (12.39 years vs. 12.77 years). All differences in information about the teachers between experiment and control schools were not significant.
*
School term on the basis of four months. 14
3.3 Instruments and data collection The data were collected using intelligence test and pupil background questionnaire for obtaining data on gender, age, parental education and occupation. The prior scores were obtained from the school archives using the same test on the same time for all the school gugus. Whereas, the content-related tests (CRT) were developed by a team consisted of teachers and university staff members. ISERP scales (Creemers, 1997; Teddlie & Reynolds, 2000) translated, validated based on pupil answers. For teaching process in the classroom, the Virgilio’s scales (Teddlie, Virgilio, & Oescher, 1990) were translated, validated, and used. Whereas for school level, related to organizational atmosphere the SLEQ developed by Rentoul and Fraser (1983) was modified and validated with an additional dimension i.e work pressure. The data came from three smaller school-cohorts, named gugus in the Sidoarjodistrict, south of the city of Surabaya, Indonesia. The gugus were Sidoarjo (as urban district), Krian (as sub-rural district) and Krembung (as rural district). Each gugus consisted of experiment schools (with treatment) and control schools (without any intervention). Table 1 shows the number of schools per gugus. The number of control schools of Gugus Krian is low. Due to circumstances it was not possible to arrange more control schools. 3.4 Data analysis Multilevel analysis was used to answer the research questions. Since the first use of multilevel analysis in School Matters (Mortimore, Sammons, Stoll, Lewis, & Ecob, 1988), then followed by many studies, it has been recognised that the statistical package of MLwiN enables more efficient estimates of school differences in pupil attainment. Because multilevel analysis is relatively new, some typical equations and explanations are presented in this section. The analysis took into account the hierarchical structure of the data in which pupils are nested within classes/schools. The multilevel analysis can be regarded as the extension of single level regression analysis by considering more than one source of variation (e.g variation of pupils in level 1 and of school in level 2). Aitkin and Longford (1986) established important bases for comparing school effectiveness after the adjustment of intake differences in individual level. The bases were regarded in the following phases of analysis.
15
3.4.1 Starting null model The first simplest one was the null model which only estimated the total variance and its components. In this model the scores were regressed on the constant term (coded 1 for every pupil). In addition the constant term was set at random at both the pupil and school levels. The aim of this analysis was to estimate the overall mean achievement at both intake and at the end of school year and also to see whether there were any school differences in mean achievement. The intra-school correlation (the proportion of the total variance which was between schools) was also computed from the random estimates. The model fitted was (based on Goldstein, 1995; Snijders & Bosker, 1999): Yij = ojxo + eoijxo with βoj = αoo + uoj
(between school variation)
where i = pupil j = school yij = pupil’s attainment xo = the intercept term (constant) with a value of 1 for every pupil αoo = overall mean attainment uoj = school level residual eoij = pupil level residual Thus the model can be summarised as follows: Yij = βo + (uoj + eoij) fixed random The following parameters were estimated by model 1: βo = overall average attainment σ2oe = pupil level variance i.e between pupils within schools variation 2 σ ou = between school variance The intra-school correlation was given by the formula: ρ = σ2ou / ( σ2ou + σ2oe) This correlation measured the proportion of the total variation that was due to schools and also the degree of similarity of the pupils within a school. The larger the value of ρ the greater the clustering and the more important it was to use a fully efficient estimation procedure (Goldstein, 1995 ). 16
3.4.2 Exploring the model at the fixed part A model reduction procedure of omitting predictors that did not contribute to the variation in achievement would refine the model into a 'minimal' model. A stepwise procedure was used in analysing the data i.e. one or a group of related variables were entered at a time starting with the simplest models and then building up to more complex models. Any predictor that did not contribute to the variance in attainment would be omitted to refine the model. Levels of significance under the 95 per cent confidence interval were obtained if the estimate was equal to or greater than double standard error (Patterson, 1991; Woodhouse, Rasbash, Goldstein, & Yang, 1996). Model 1 was extended by the inclusion of explanatory variables measured at pupil, class and school level. The purpose of fitting this model was to find out which of the pupil, class and school factors had significant effect on attainment. The model was represented by the equation: Yij = βoij xo + β1 x1ij + ... + βn xnij with βoij = βo + uoj + eoij where i = pupil j = school Yij = response variables x1ij ... xnij = explanatory variable (eg gender, age, etc) uoj = school/class level residual eoij = pupil level residual β0 = constant (intercept term) β1 ... βn = regression coefficient Some models were developed using the equation above. After performing the null model, the modelling stages dealing with explanatory variables were consecutively entered: initial scores as baseline, child background and characteristics, class and school variables. Then the variances attributable to school and pupil were checked. 3.4.3 Identifying the quality improvement The third stage was the identification of school success for each teaching unit using the residuals estimates and their associated confidence limits which pinpointed those schools 17
performing better or worse than expected from their intake (p
