ICERE 2016
EVALUATING POLICY IMPLEMENTATION INDICATORS IN DECENTRALIZED SCHOOLS Lilik Sabdaningtyas1 and Budi Kadaryanto2 1Department
2
of Educational Sciences, University of Lampung Department of Language and Arts Education, University of Lampung
[email protected]
Abstract— Education decentralization in Indonesia has been initiated following the commencement of the reform era in the beginning of the new millennium which is resembled in a School Based Management (SBM) system. This brought impact on the schooling system as well, shifting the school management from the centralized into the decentralized one. The purpose of this study is to analyze the dynamic factors considered contributing to the performance of policy implementation of decentralization at the school level. In addition, it further evaluates those factors that serve as indicators for the schools performance in implementing decentralization at the school level. In a proportionally stratified random sampling, 182 respondents involved in the management of their school were asked to fill out the questionnaires about school climates, school policy executives, school policy targets, school resources as well as school organization management. This study was carried out in junior high schools in an urban area in Lampung Province. In a Structural Equation Modelling analysis, the result of the study are promising, as this indicates distinctive spectrum for evaluating the policy implementation for decentralized school management and its contributing indicators. These five indicators could be utilized as a means of evaluation how a school implements decentralization policies and as key performance indicators for their policy implementation. It further proposes a broader support for the reform among school principals, board members and related stakeholders to optimize their performance in implementing SBM. Keywords: education decentralization, evaluation, policy, School Based Management
I.
INTRODUCTION
In the beginning of the new millennium, the decentralization system in the government of Indonesia has brought the authority to school levels to manage and run their administrative activities in more democratic ways. This has been believed as an emerging phenomenon in most education systems in other countries [1][2][3]. Education decentralization has been associated with an act of giving more space for schools to improve student outcomes and the effectiveness of school systems by the transfer of authority. In other words, it was massively endeavored by practitioners, academicians, stakeholders, as well as schools to quest for the central government to devolve power and authority to even the lowest authority at the school level comprising school administrators, teachers, parents, as well as the community. At one hand, it is assumed that this situation brought euphoria for schools to develop and to run a democratic system, in which this had also existed in most countries during that period, those questing for the decentralization to occur at the school level had to 1
ICERE 2016
be aware of the impact of turmoil of the new reform era turnover. This has been warned by Louis [ 4 ] who suggests that educational reform (moving from centralized to decentralized one) is not a movement of no cost and difficulties as each level of schools has to develop their schools on their own. Louis further suggests that such reforms in education require extensive, consistent support, accompanied by in-service training and technical assistance for school leaders – enabling them to change management and planning skills, and helping them to deal with the school and classroom implications of reforms [5][6][7]. The implementation of School Based Management (SBM) in Indonesia has been always marked as educational reforms, not only driven by the in country turmoil as the fall of new order, and entering the reform era, but this movement (SBM implementation) has actually been driven by many successful countries bringing a fundamental change in the school management and administration. SBM requires a school as the homefront of formal education strata to have a decision-making authority that stimulates and sustains improvements at the school. In the meantime, the degrees of school authority in making decisions in the domains of the school’s mission, goals and school policies relating to financial, material and human resources should not simply delegated to them, rather transferring the authority to a representative managerial body called the school council or board (Komite Sekolah). With this way, there will be a synchronous actions within the school community, which may lead to have all parties’ commitment to making school visions real under the umbrella of SBM [8][9][10][11] The government decision in succeeding the policy of SBM implementation has been partially driven by the belief that community control in the school management and administration will make schools become more effective and efficient in developing their visional programs. This idea came about in the United States in the mid-17th century. However, this idea had gained better and more widespread acceptance in 1980s, where the concept of community participation in SBM has become a major research themes and has been regarded as a policy initiative in school reforms in many education systems in the Europe and some parts of Asia. The Indonesian government itself was 2
ICERE 2016
found a bit later in comparison to other Asian countries like Japan, China and Thailand in showing interest in implementing SBM [12]. The idea of the SBM implementation should remain in public control while simultaneously fostering good instructional practices and good management tactics, including the prospect of effective accountability to all stakeholders. These decentralization initiatives take many forms, including the empowering of principals, teachers and parents. Today, educational decentralization with devolution of authority to individual institutions is a popular reform theme of governments around the world [13]. The government policy in making the transition from autocratic to democratic forms of government, has truly brought great impact in decentralizing educational systems in the country. This situation results in gaining citizen participation in government institutions. Winkler suggests that improving the quality of education is often offered as a goal of decentralization, reflecting the notion that local people can solve local education problems better than the centralized state system [14]. The success of SBM implementation will be the government challenge to assure the process run as it supposed to be. A policy will just be a dream that is stored as an archive alone if not implemented effectively. Thus the policy implementation is an important aspect that can determine the success or failure of a policy implementation in educational aspects [15]. To determine the success or failure of policy implementation, it requires evaluative actions, as this will give portray of how the policy has been implemented. Once the implementation of policies that are not evaluated, then there will no information about the success of the implementation. This statement indicates the importance of the policy implementation evaluation itself and it crucial ties. For the purpose of policy evaluation, it is required that proper evaluation model to evaluate the results to be accurate. Therefore, it is necessary to develop appropriate evaluation model to measure the performance of the implementation of the decentralization policy in the educational unit (schools). The purpose of this study is to propose an evaluation model to evaluate the performance of the implementation of the decentralization policy in the 3
ICERE 2016
educational unit (schools). In a theoretical concept, achieving policy objectives (policy output) can be approached by actions that form the so-called resource mobilization policy input (input policy) and so-called resource management policy cycle (policy cycle at the stage of formulation, implementation, and policy evaluation [16]. This study focuses only on the stages of implementation of the policy (policy implementation). Furthermore, to produce a performance that effective policy implementation will require dynamic factors consisting of: environmental policy, implementing policy, group policy objectives, and resources, which is the fourth and policy input will be instrumental in the implementation of the policy cycle [17]. Based on the theoretical concepts, a model developed by referring to a model that uses a system of thought patterns and theoretical evaluation of policy implementation among a few researchers [18][19][20][21][22][23]. They stated that the matching performance of policy implementation should be supported by a variety of factors covered by a policy system. Based on the opinion of experts, the factors supporting policy implementation performance in this study was developed by focusing on components of: input policy, cycle policy, and outputs policy. It is necessary to develop appropriate evaluation model
to
measure
the
performance
of
the
implementation
of
the
decentralization policy in the educational units (schools). II.
METHOD
The design of this study is research & development (R & D) that utilizes a quantitative approach in a non-experimental design. This so-called evaluation models of policy implementation performance can be categorized as a study that utilizes a formal evaluation approach with a retrospective evaluation process. This kind of research tends to rely on the description of ex post facto (retrospective) about the ongoing program of activities, which in turn relates to the output as well as impacts of a policy implementation [24]. Therefore, the design of such studies will not employ treatments or experiments by the researcher.
4
ICERE 2016
The sampling method used in this study was a proportional stratified random sampling technique. With this way, before selecting a random sampling technique, it should be initiated by determining the minimum number of the sample by using a minimum sample calculation formula of Cohen [25]. Based on Cohen's formula, power determined would be 60; with a = 0.01; R2Y.B = 0.10; and u = 4. Therefore, the samples obtained in this study were at least 13 schools consisting of five state junior high schools and eight private junior high schools in Bandar Lampung. The variables employed in this study consisted of both latent variables and observed variable. They were categorized as follows 1). School climate consisting of three observed variables; 2). School policy executives that consist of three observed variables. 3). School policy target consisting of also three observed variables. 4). School resources with four observed variables. 5). School organization management with three observed variables. The data were collected by using questionnaires filled out on three groups of respondents namely the principal, teachers, and school committees as the policy executives at the school level as a result of decentralization in education. The instrument used consists of five items composed inventory each designed to get the data on school climates, school policy executives, school policy targets, school resources as well as school organization management, the target group of school policies, and school resources. A total of 115 items were developed by using Likert scale with four alternative answers. To determine the validity of the instrument, construct validity as well as content validity were used. The reliability of the instrument was calculated by using "Cronbach Alpha" with coefficient reliability of at least 0.5. Meanwhile, the data analysis was conducted in two phases namely: analysis of test requirements to test the normality and multikoleniaritas data, and test hypotheses to see the model fit using SEM with LISREL program [26]. This research and development of model of current study adopted the model proposed by Cennamo & Kalk that was conducted in two phases namely: The first phase Instrument Development / Pre-Development of the steps which were carried out in the following steps; 1) theoretical development, 2) describing the 5
ICERE 2016
theories into components, variables, and indicators in the form of table specification for the instrument development, 3) developing opaque instruments to be validated by using the Delphi technique policies, 4) developing instruments, as well as 5) testing the instruments. The test instruments were analyzed by using SPSS version 15.0. The test shall be imposed on the number of respondents from Junior High Schools 4 to 56 people consisting of 4 principals, 32 teachers, and 20 school committee. 6) The analysis of the results of testing instruments was carried out by using exploratory factor analysis (EFA). If the test turns out to be empirically valid and reliable, then the instrument developed can then subsequently be used as a tool to develop a theoretical model of the proposed research [27]. The second stage of this research and development is development itself. The development of an evaluation of policy implementation was referring to model development steps proposed by Ferdinand [28], Narimawati [29], and Ghozali [30]. The model development step employed in current study were: 1) theoretical/conceptualization of the model developed. 2) Developing of flow charts. 3) Converting the flow chart into a series of structural equation and measurement models specifications. 4) Selecting the estimation technique for the model to be created. 5) Product try out. The product try out was carried out to test the suitability of the products made by theoretical models with empirical models. Testing the suitability of the model imposed on 13 Junior High School respondents in Bandar Lampung that implement decentralization policies at the high school level education units comprising principals, teachers, and school committee. 6) Evaluation of the model. Prior to the evaluation of the model, first evaluation requirements such models is the sample size, normality test, and test multicollinierity. The next step carried out was evaluating the model fit (goodness of fit) by using the alignment criteria. Alignment criteria used is, Chisquare (X²) and probability (p), Root Mean Square Error of Approximation (RMSEA), normed Fit Index (NFI), Comparative Fit Index (CFI), Incremental Fit Index (IFI), Relative fit Index (RFI), Goodness of fit Index (GFI), Adjusted Goodness of fit Index (AGFI).
6
ICERE 2016
III.
RESULTS AND DISCUSSION
Results Overall, the instrument of performance of policy implementation indicated that there were no items that had communalities 1, this figure shows that the factors raised can be used as an indicator of some traits. The reliability of the whole factor was found more than 0.70, meaning that all items can be declared as the reliable indicators. In we take a look at the school organization management, the data show that the cumulative percentage was found good enough for the three factors which was equal to 69.32%. This resembles that the instrument of school organization management could explain the theory of three-dimensional factor of as much as 69.32% with the Eigenvalue reaching to 1.286 > 1. This figure shows that the factors raised can be used as an indicator of the traits. Meanwhile, the common factor was found to have no items smaller than 0.45, meaning that all items can be interpreted. Further, the reliability of the whole factor was also found more than 0.70, meaning that all items can also be declared as the reliable indicators. Subsequently, the result of the school climate instruments shows that the common factor (communalities) was found to have no items smaller than 0.45 meaning that all items can be interpreted. Judging from the cumulative percentage, the finding was good enough for the three factors with the percentage of 76.34%. This means that the instrument for evaluating the policy implementation about the school climate could explain the dimensional theory of three factors as much as 76.34% and the Eigenvalue was found to be 1.462> 1. This figure shows that the factors proposed in the instruments can be used as indicators of the traits. The reliability value of the whole factors was found more than 0.70, meaning that all items can be declared as the reliable indicators as well. 7
ICERE 2016
In the school policy executives’ factor, the data show that the common factor was found to have no items smaller than 0.45 meaning that all items can be interpreted. Judging from the cumulative percentage, the figure was found to be good enough for the three factors which was equal to 85.08%, making the instrument of school policy executives can explain the theory of threedimensional factor of as much as 85.08% with the Eigenvalue for 1,045> 1. This figure shows that the factors raised can be used as the indicator of the traits. The reliability of the whole factor was also found to be more than 0.70, meaning that all items can be declared as the reliable indicators as well. In the school policy target, experimental data show that the common factor was found to have no items smaller than 0.45 meaning that all items can be interpreted very well. The cumulative percentage was also found good enough for the three factors which was equal to 76.52%, and making the instrument for the school policy target can explain the theory of three-dimensional factor of as much as 76.52% with the Eigenvalue of 1, 045> 1. This figure shows that the factors proposed can be used as an indicator of the traits. The reliability of the whole factor was also found to be more than 0.70, meaning that all items can be declared as the reliable indicators as well. In the meantime, the school resources data show that the common factor indicated that there was only one item whose value is smaller than 0.45 with the value of 0.397. Other items had common factor value to be more than 0.45, meaning that most of the items can be interpreted. Judging from the cumulative percentage, it was also found good enough for the four factors that was equal to 66.98%. This means that the instrument can explain the dimensional theory of the four factors which were found to be more than 66.98% with the Eigenvalue 1.053> 1. This figure shows that the factors raised can be used as an indicator of the traits. The reliability of the whole factor was also found to be more than 0.70, meaning that all items can be declared as the reliable indicators as well. The data normality test results was found to have abnormal distribution for the manifest y9, Y11, x10 and x11. This is characterized by a chi-square value with a probability of less than 0.05. About the manifest of the abnormal distribution, normalization of the data was carried out [ 32 ]. After the 8
ICERE 2016
normalization test, it shows all manifests have chi-square value with a probability of more than 0.05, then it's normal. Normality test results are all manifests; Y4 = 0.078; Y5 = 0.118; Y6 = 0.122; Y7 = 0,090; Y8 = 0.055; Y9 = 0.997; Y10 = 0, 074; Y11 = 1.000; Y12 = 0.104; Y13 = 0.110; Y14 = 0.087; Y15 = 0.109; Y1 = 0,140; Y2 = 0.075; Y3 = 0.117; X1 = 0.175; X2 = 0.067; X3 = 0.082; X4 = 0.077; X5 = 0.098; X6 = 0.200; X7 = 0.096; X8 = 0.162; X9 = 0.080; X10 = 0.998; X11 = 0.999; X12 = 0.375; X13 = 0.067. The multicollinearity test was intended to see whether there is a perfect correlation or large value among the independent variables. If there is a magnitude of correlation coefficients more than 0.80, it means that the multicollinearity is present. Based on the test results of the data multicollinearity, it turns out that the correlation among the dependent variables was nothing found more than 0.80. This means that there is no multicollinearity of the data.
Figure 1. The evaluation of policy implementation model [33]
The evaluation of policy implementation model can be seen in Fig. 1. The test results of the proposed model has a goodness of fit index parameter which does not meet the entry requirements model of the chi-square = 534.2862 (not met), p = 0.000 (not met), and RMSEA = 0.06149 (not met). Then later model was modified by correlating between the two errors indicators that have large residual covariance. The result of these modifications successfully reduced chi9
ICERE 2016
square and probability values, so that the model becomes suitable. Fig.2 is modeled after the modification. Apparently the result of the calculation of the Goodness of Fit Index has met the criteria. Calculation of the model is that the modified chi-square = 370.048 (met), p = 0.07363 (met), and RMSEA = 0.031 (met). The results of this study can be interpreted that the model developed is received after modified.
Figure 1. The evaluation of policy implementation model [34]
Discussions These findings support the theory of policy implementation model proposed by several previous studies such as by Grindle [35] which states that any implementation of the policy needs to consider the context or environment (school climate) in which the action (action) is performed. Referring to the opinion of Grindle, it means that school climate is an important determinant of performance policy implementation. Mazmanian and Sabatier argued that the implementation of the policy will be effective when the implementation comply with what has been laid down by the regulations (technical instructions, implementation guidelines). Further, they stated that the target group must also comply with the program, regardless of their compliance with the policy objectives whether or not achieved. Referring to the opinion of Mazmanian and 10
ICERE 2016
Sabatier, it means that the target groups and implementing policies is also an important determinant of performance policy implementation [36]. Edwards argues that one of the variables that affect policy implementation is resources (resource). In this case necessary for the implementation of a policy is not only about the staff number / personnel sufficient but preferably expertise or skills in implementing policies that have been formulated. Edwards refers to the opinion that the quality and quantity of resources, especially human resources is a factor supporting the implementation of the policy [ 37 ]. Furthermore, according to Smith there are four important components in the implementation of the policy; the idealized policy, the target group, the implementing organization, environment factors. On the basis of Smith's opinion that the target group policy, implementing policy and environmental factors are also factors supporting the policy implementation [38]. Meter & Horn suggested that the performance of the implementation of the policy can also be determined by factors; standards and targets, resources, communication between organizations, the characteristics of the implementing organization, and the social, economic and political [39]. These five factors will shape attitudes towards implementing policies that will be implemented and ultimately affect the performance of the policy implementation. Cheema & Rondinelly
suggests
four
factors
that
are
thought
to
influence
the
implementation of decentralization policies suggested as the independent variable; environmental conditions, the relationship between the organization, resources, and the character of the executing agent [40]. Implications of the acceptance of this theoretical model is expected to be followed-up for the program manager to conduct training on the performance of schools in implementing the decentralization policy of compulsory education schools that implement the functionality. The policy implementation for SBM in Indonesia has been largely educational reforms following the reform era. This movement (SBM policy implementation) has actually been driven by many successful countries bringing a fundamental change in the school management and administration. SBM requires a school as the homefront of formal education strata to have a decision-making authority that 11
ICERE 2016
stimulates and sustains improvements at the school. In the meantime, the degrees of school authority in making decisions in the domains of the school’s mission, goals and school policies relating to financial, material and human resources should not simply delegated to them, rather transferring the authority to a representative managerial body called the school council or board (Komite Sekolah). With this way, there will be a synchronous actions within the school community, which may lead to have all parties’ commitment to making school visions real under the umbrella of SBM. This has been regarded as the the foundation of the SBM policy implementation success. One of which way has been proposed to be evaluated by utilizing the policy implementation performance indicators discussed above
Conclusion The current study is trying to propose an evaluation model that evaluates the performance of a policy implementation in the context of education decentralization at the school level.
The evaluation of policy objectives
achievement (policy output) can be approached by actions, either in the input policy level, policy implementation, or the management policy level (formulation, implementation, and policy evaluation). This study focuses on the stages of implementation of the policy (policy implementation). School climates, school policy executives, school policy targets, school resources as well as school organization management were found applicable to be utilized for evaluating the policy implementation for decentralized school management and its contributing indicators. It gives implication for a broader support for the reform among school principals, board members and related stakeholders to optimize their performance in implementing SBM. REFERENCES [1]
Patrinos, H. A., & Fasih, T. (2009). Decentralized decision-making in schools: The theory and evidence on schoolbased management. World Bank Publications.
[2]
Rado, P. (2010). Governing decentralized education systems: Systemic change in South Eastern Europe.
[3]
Gershberg, A. I., González, P. A., & Meade, B. (2012). Understanding and improving accountability in education: A conceptual framework and guideposts from three decentralization reform experiences in Latin America. World Development, 40(5), 1024-1041.
12
ICERE 2016
[4]
Louis, K. S. (1998). " A light feeling of Chaos": Educational reform and policy in the United States. Daedalus, 127(4), 13-40.
[5]
Woods, D. (2002). Moving Forward... From Where You Are to School Improvement That Lasts: A ResearchBased Guide.
[6]
Fiszbein, A. (Ed.). (2001). Decentralizing education in transition societies: Case studies from Central and Eastern Europe. World Bank Publications.
[7]
Gamage, D., & Sooksomchitra, P. (2006). Decentralisation and school-based management in Thailand. In Decentralisation and Privatisation in Education (pp. 151-167). Springer Netherlands.
[8]
Caldwell, B. J. (2005). School-based management (Vol. 3). Paris: International Institute for Educational Planning.
[9]
Sumintono, B. (2006). Decentralized centralism: School based management policies and practices at state secondary schools in Mataram, Lombok, Indonesia.
[10] Patrinos, H. A., & Fasih, T. (2009). Decentralized decision-making in schools: The theory and evidence on schoolbased management. World Bank Publications. [11] Bandur, A. (2009). The implementation of School-Based Management in Indonesia: Creating conflicts in regional levels. Journal of NTT Studies] Vol, 1(1), 17. [12] Shoraku, A. (2009). Educational movement toward school-based management in East Asia: Cambodia, Indonesia and Thailand. Background Paper for EFA Global Monitoring Report. [13] Hanson, E. M. (1998). Strategies of educational decentralization: Key questions and core issues. Journal of educational administration, 36(2), 111-128. [14] Winkler, D. R. (1993). Fiscal decentralization and accountability in education: Experiences in four countries. J. Hannaway y M. Carnoy (Comps.), Decentralization and school improvement: Can we fulfill the promise, 102-134. [15] Udoji, C. J. (1981). The African public servant as a public policy maker. Public Policy in Africa, Africa Association fo Public Administration and Management. Addis Abeba. [16] Dunn, W. N. (2015). Public policy analysis. Routledge. [17] Mustopadidjaja, AR. (2002). Manajemen proses kebijakan publik. formulasi, implementasi, dan evaluasi kinerja. Jakarta: Lembaga Administrasi Negara. [18] Grindle, M. S. (Ed.). (1980). Politics and policy implementation in the Third World (Vol. 310). Princeton, NJ: Princeton University Press. [19] Mazmanian, Daniel., & Sabatier, Paul. (1986). Effective policy implementation. Lexington: D.C, Heath [20] Ingram, H. M., & Mann, D. E. (Eds.). (1980). Why policies succeed or fail (Vol. 8). SAGE Publications, Incorporated. [21] [38] Smith, T. B. (1973). The policy implementation process. Policy Sciences, 4(2), 197-209. [22] Van Meter, D. S., & Van Horn, C. E. (1975). The policy implementation process a conceptual framework. Administration & Society, 6(4), 445-488. [23] Rondinelli, D. A., Nellis, J. R., & Cheema, G. S. (1983). Decentralization in developing countries. world bank staff working paper, 581. [24] Dunn, William N. (1998). Pengantar analisis kebijakan publik (ed II). cetakan I. (terjemahan Samodra Wibawa, dkk). Yogyakarta: UGM Press. (Buku aslinya diterbitkan tahun 1994). [25] Cohen Jacob. (1977). Statistical Power Analysis for the behavioral Sciences. New York San Francisco London: Academic Press, A Subsidiary of harcourt brace Jovanovich, Publishers [26] [28] Ferdinad, A. 2000. Structural Equation Modeling Dalam Penelitian, Badan Penerbit Universitas Diponegoro, Semarang [27] Cennamo, K., & Kalk, D. (2005). The professional designer. Real world instructional design, 272-285. [29] Narimawati, U. (2007). Riset Manajemen Sumber Daya Manusia. Jakarta: Agung Media. [30] Ghozali, I. (2005). Model Persamaan Struktural. Semarang: UNDIP.
13
ICERE 2016
[31] Nurosis, M. J. (1993). SPSS. Statistical data analysis. Spss Inc. [32] Setyo Heri Wijayanto. (2008). Structural equation model. Yogyakarta: Graha Ilmu. [33][34] Lilik Sabdaningtyas. (2012). Model Teoritik Evaluasi Kinerja Implementasi Kebijakan. Makalah Proceeding Konferensi Ilmiah Nasional. Surabaya: Kampus Unesa. [35] Grindle, M. S. (Ed.). (1980). Politics and policy implementation in the Third World (Vol. 310). Princeton, NJ: Princeton University Press. [36] Mazmanian, Daniel., & Sabatier, Paul. (1986). Effective policy implementation. Lexington: D.C, Heath [37] Edward III., George C (1980). Implementing public policy, conggressional quarterly. Washington, D.C. Ferdinand, Augusty. (2000). Structural equation modeling dalam penelitian manajemen. aplikasi model-model rumit dalam penelitian untuk tesis S2 dan disertasi S3. Semarang: Badan Penerbit Universitas Diponegoro. [39] Meter, D.S., & Horn, C.E. (1973). The policy implementation process a conceptual framework in administration and society. Baverly Hills: Sage Publication. [40] Rondinelli, D. A., Nellis, J. R., & Cheema, G. S. (1983). Decentralization in developing countries. world bank staff working paper, 581.
14
