Stigmas without Vouchers: The Impact of Ratings on ...

Stigmas without Vouchers: The Impact of Ratings on Student Achievement in Florida’s LowPerforming Schools

Daniel H. Bowen University of Arkansas Julie R. Trivitt Arkansas Tech University

Prepared for the Association for Education Finance and Policy Annual Conference (March 2012)

Abstract Under the Florida A+ Accountability Program, Florida’s schools are graded based on student performance on the Florida Comprehensive Assessment Test (FCAT). Previously, when schools would earn their second failing grade within four years, students assigned to these schools were offered vouchers to attend private schools. However, in January of 2006 the Florida Supreme Court declared the private school voucher component of the A+ Program unconstitutional, eliminating the private school voucher threat for students enrolled in failing schools. This exogenous shock allows us to test whether private voucher threats and the funding tied to students matter, or whether it is the stigma of publicly receiving a failing grade that motivates schools to respond. We find no evidence that the private school voucher threats drove academic improvement beyond what is seen in failing schools when this voucher threat is removed. Working Draft – Please Do Not Cite

Introduction When A Nation at Risk was published in 1983, America lost the luxury of operating under the presumption of educational excellence in our public schools and was forced to reconsider many aspects of our educational framework. In the 29 years since, school systems have addressed lagging student achievement with a plethora of education reforms and accountability policies. No Child Left Behind (NCLB) made accountability-based standardized tests (high-stakes testing) universal in 2002. While not exactly the same, much of NCLB was modeled after Florida’s A-Plus Accountability Program (A+) which was implemented in 1998. Under A+, schools are graded based on student performance on the Florida Comprehensive Assessment Test (FCAT). When a school earned its second F within 4 years, students assigned to that school were offered vouchers to attend a private or another public school of their choice. Initially, the percentage of a school’s students earning level 2 (out of 5) or better on the reading, writing, and math portions of the exam would determine its grade. For the 2001 – 2002 school year the school grading scheme converted to a point-accumulating system, where schools now earn points based on students’ achievement levels as well as the percentage of students who make learning gains during the academic year. After the first few years of the program no schools received two failing grades to trigger the automatic voucher offers for their students (Florida Department of Education, 2008). When faced with an accountability program, schools found ways to make sure student achievement was high enough to avoid failing grades for the school, and indeed the overall distribution of school grades improved each year. Several studies (Figlio & Rouse, 2005; Greene, 2001) have documented statistically significant gains for students in F schools relative to their peers in D schools. Schools clearly responded to the accountability regime. Accountability proponents saw the voucher sanction and corresponding financial incentives as the key to success in A+ (Greene, 2001; Greene & Winters, 2003; West & Peterson, 2006). Others attribute the results to more shame-based motivations, arguing that the stigma of teaching/attending a failing school or recognition of relatively poor quality served to trigger changes when the status quo was no longer acceptable (Figlio & Rouse, 2005; Goldhaber & Hannaway, 2004; Ladd & Glennie, 2001).

2

In January of 2006 the Florida Supreme Court declared the voucher component of A+ unconstitutional, immediately eliminating the voucher threat for students of failing schools in the accountability program (Bush v. Holmes, 2006). This immediate change allows us to test whether it is the private school voucher threat and the funding tied to student counts or a matter of pride that ultimately motivated schools to respond to A+. We find no evidence that the financial threat associated with a private school voucher program produced academic improvement beyond what is seen in failing schools when this sanction is removed.

Data This project utilizes a dataset provided by the Florida Department of Education containing test scores in reading and math as well as demographic information for all students enrolled in Florida public schools in grades 3 – 10 from 2002 – 2008. The student level data is supplemented with school level information on the point total and grade earned by each school over the same time period. This dataset consists of over 8 million student/school/year observations. For this analysis we focus on a small subset since we are utilizing a regression discontinuity (RD) model. We use only students in grades 4 – 6 attending schools that earned a grade of D or F within 0.5 standard deviations of the D/F break point. Literature Review In one of the earliest evaluations of A+, Greene (2001), using a basic RD design, finds that higher-scoring F schools significantly improved in student achievement relative to lowerscoring D schools. On average, students attending the higher-scoring F schools improved by 0.12 standard deviations in reading, 0.30 in math, and 0.41 in writing. 1 Figlio and Rouse (2005) apply a school-level fixed effects estimation that produces fairly similar results. Their research finds that schools threatened with voucher assignments were making larger scale achievement gains in reading and math, approximately 0.10 and 0.25 standard deviations respectively. However, these impacts were much smaller when estimating the effects on lower-stakes, nationally normreferenced tests.

1

Math and writing results significant at the p < 0.01 level.

3

West and Peterson (2006) evaluate the effects of the changes and adaptations of A+ that took place in 2002. The new grading policy put in place along with the implementation of NCLB provided an exogenous shock to measure the performances of newly designated F schools as well as those identified as failing to meet adequate yearly progress (AYP). They conclude that schools newly designated as F schools as a result of the change in grading policy performed at a significantly higher level than if they had avoided the sanctioning. Finally, Chiang (2009) finds that the general threat of sanctions under A+ produced increases of 0.11 standard deviations in reading as well as 0.12 standard deviations in math.2 Other research on this matter examines school-level responses to A+. Goldhaber and Hannaway (2004) conduct a qualitative, survey-based study of five of Florida's public schools. They find that school personnel are more conscious of the social stigma that accompanied their schools' grades than with the possibility of students acquiring vouchers. While the nature of these school-level responses does not exactly align with our study, it is worth noting that this research may help to explain the influence the stigmas carry at the school level. Rouse et al (2007) perform a mixed-methods analysis that combines a RD design in tandem with a survey of Florida's schools, exploring schools' responses to A+ accountability pressures and sanctions that may not appear in state-collected data. They find that schools were in fact changing their practices in meaningful ways (e.g. giving more attention to low-performing students, lengthening the amount of time devoted to instruction, etc.). Additionally, they uncover from the survey data that these changes were likely attributable to changes in school policies and practices (e.g. restricting administrators "zone of acceptance"). Chakrabarti (2007) examines how F schools were reacting to receiving their first failing grade. This study concludes that there was strong evidence of schools responding to the incentives imbedded in A+. After receiving their first grade of F, schools were placing more focus on lower-performing students without actually harming higher-performing students' achievements. These schools also were often devoting more time and attention to the writing section of the FCAT in order to avoid a subsequent grade of F. Schools have to fail all three sections of the test in order to receive a grade of F (Florida Department of Education, 2002). 2

Reading found significant at the p < 0.10 level while math significant at the p < 0.05 level at the preferred (28 point) discontinuity bandwidth.

4

Therefore, schools at risk of earning a second F would often focus on making gains in the writing section to assure obtaining no lower than a D. A major, unresolved issue has been whether the private school voucher sanction (or threat) was having an independent effect on student achievement. Ladd and Glennie (2001) apply Greene's (2001) research design to North Carolina data. Although North Carolina had a comparable accountability system to Florida in 2001, the biggest difference in the two states' systems was that North Carolina lacked a voucher sanction. Ladd and Glennie find that, like Greene, schools identified as relatively higher-scoring, low-performance schools (i.e. analogous to the relatively higher-scoring F schools in Florida) produce greater achievement increases relative to the relatively lower-scoring, below average performance schools (i.e. comparable to the lower-scoring D schools in Florida). Ladd and Glennie conclude that the comparable levels of achievement gains in North Carolina demonstrate that Florida's voucher sanction was not a necessary component for increasing student achievement. Figlio and Rouse (2005) evaluate the initial implementation of the A+ voucher sanction to determine whether bringing vouchers into the accountability system significantly improves student achievement. Florida's schools were not graded prior to A+, but the lowest performers were still labeled as "critically low" prior to the application of school grades. 3 They find that the addition of the voucher sanction did not significantly improve student performance, suggesting that other sanctions, namely the stigmatizing of poor-performing schools, have greater influence over these improvements in achievement. Contrary to both Ladd and Glennie and Figlio and Rouse, Greene and Winters (2003) conclude that private school vouchers were a significant component of A+ in increasing the performances of low-performing schools. Greene and Winters inspect the differences in levels of voucher threat to determine whether this sanction influenced school performance. They discover that schools experiencing greater pressures (i.e. schools already sanctioned or one F away from sanctioning) were producing greater gains relative to similar schools not feeling the same "heat" (i.e. those schools with the threat lifted from going three years without a subsequent F or those that remained at the D level). They find that the differences in achievement gains, directly related 3

The authors argue that "critically low" performance is analogous to receiving a grade of F prior to A+.

5

to extent of voucher threat, serve as evidence for the voucher threat independently driving at least some of the gains observed under A+. West and Peterson (2006) come to a similar conclusion as Greene and Winters (2003). However, their study differs in that it compares the influences over Florida's student achievement brought about by NCLB choice sanctions versus the accountability measures of A+. They find that public school choice stemming from failure to meet Adequate Yearly Progress (AYP), under NCLB, was not independently producing results comparable to those experienced with the private-school vouchers issued under A+. They conclude that private schools vouchers have played a significant, independent role in increasing student achievement levels in lowerperforming schools. Estimation Strategy Our methodology for studying the effect of the F sanction on student achievement is similar to the Rouse et al (2007) strategy. Since the point system used to grade schools changed the same year as the voucher threat was removed, we are unable to use a cubic function of the schools’ point totals to account for school quality. To measure only the effect of a school receiving an F in the prior year and minimize picking up regression to the mean tendencies, we employ a regression discontinuity approach and limit the sample to students in schools that were within 0.5 standard deviations of the D/F point breakpoint. For the 2002 – 2005 school years in our dataset, this includes D-graded schools earning between 280 and 305 points, and schools earning an F with 254 – 279 points. The school grading scale changed for the 2006 school year, so for later years D-graded schools are included if they earn 395 – 428 points and F-graded schools are included when earning 362 – 394 points. This leaves us with several thousand student-level observations from which we estimate the following model: (1) Tist = β0 + β1f(Tist-1) + β2Xit + β3 Yt + β4Fst-1 + β5 F*Threat + εst. Where T indicates the test score for student i in school s during year t; f(Tist-1) is a cubic function of the students test score in the prior school year; Xit is a vector of observed student 6

characteristics in year t, Y is a year indicator (the calendar year of the fall semester), Fst-1 indicates the school attended by student i in year t received an F in the previous school year; F*Threat indicates the school earned an F in the previous year and the private school voucher threat was still present, and εst is a stochastic error term clustered by school and year. Tables 1-3 display descriptive statistics for these data by grade. We estimate this model for each grade level from 4 to 6 for reading and math test scores. Grade 3 is excluded because it is the first grade level when scores are available, so no previous years are available for prediction. Grades above 6 are excluded since there were so few junior high and high schools earning F grades under A+. We estimate a separate equation for each grade to allow for different achievement growth patterns by grade level as the curriculum, test difficulty and other cohort level conditions vary. The coefficients of interest are β4, which estimates the overall effect of receiving a high F as opposed to a low D, and β5, which estimates the differential effect of receiving an F when voucher threats are still present. Results The results from estimating the model using math test scores are in Table 4. As expected previous test scores and student demographic controls have a highly statistically significant influence on current test scores for every grade level. 4We estimate two models, one with 2008 included and one with 2008 excluded.5 In all models the excluded year for the year indicator is the 2005-2006 school year. This is the year during which the Florida Supreme Court ruling (Bush v. Holmes, 2006) was announced and the year that the accumulating point total grading system was changed. For math scores, the coefficient on the school receiving an F in the previous year is positive and statistically significant in 4 of the 6 equations, indicating that students in schools recently earning a high F do indeed see larger test score gains that students in schools recently earning a high D. The coefficient on the interaction term for a school earning an F while the threat was in place is negative but statistically insignificant, indicating the threat of students 4

One interesting result is that race does not seem to matter, at least in math test scores, in the grade 6 equation. Florida implemented a change in their school grading procedure at this time. Science test scores and an emphasis on the gains of lowest performing math students were now part of the grading system. Therefore, we include additional specifications to examine whether excluding this year affects results. 5

7

receiving private school vouchers has no additional impact beyond the stigma of being a failing school. The results obtained from the data on reading test scores are in Table 5. As in the math results, previous test scores and demographic controls were statistically significant and in the direction consistent with other results. Again, we see positive results when the school earned an F in the previous year; although, they are only statistically significant in 3 of the 6 equations. When the F grade is interacted with the existence of the voucher threat, the coefficients are negative, have an absolute value greater than the F coefficient, and are statistically significant in 4 of the 6 equations. This result would suggest that the threat of private school voucher sanctions actually contributed to lower student test performance in reading rather than higher. Conclusions These results confirm the conclusion of Figlio and Rouse (2005) that the private school voucher sanction of A+ did not independently bring about increases in achievement for students enrolled in low-performing schools. In the majority of our model specifications, elementary students attending schools that recently received grades of F make statistically significant gains in both math and reading. However, when examining the interaction of a school failing grade with the presence of a private school voucher sanction (threat), we find no impact on math scores and actually a statistically significant decrease in reading. While not immediately obvious, there are a few possible explanations for the seemingly illogical findings. First, the lack of effect for sanctions in math could indicate that the possibility of losing students through private school vouchers does not motivate schools, but these schools are motivated to remove the stigma of a failing grade. However, this does not explain the negative effect of sanction threats on reading performance. Our findings could indicate that school improvements, implemented as a result of accountability initiatives, have a positive effect that grows over time. If student performance is improving over time as a result of these accountability measures, test scores would be highest in the most recent years. Since this timing coincides with when the voucher threat was removed it would signify higher student performance in the absence of voucher threats. 8

Another possible explanation is that something unobservable changed when the voucher threat was removed, such as parental attitudes or involvement. It is plausible that parents lose the motivation to become more active and work with these struggling schools to improve academic performance. These parents may view another failing grade for their school as a means out of an underperforming school that comes in the form of a voucher. However, once the voucher program is eliminated, parents who care about academic performance now have stronger incentives to become involved and demand that their child’s school improves immediately rather than waiting to see if the voucher escape route materializes. If more active parents are able to initiate changes that improve school performance more than the changes in response to private school voucher threats, particularly changes that enhance reading test scores, we could see voucher threats showing up as being less effective than the world without voucher threats. Such a result would occur because of highly effective, though unobserved, active parents. Moreover, the fact that this effect is found with reading may also feed into this interpretation since reading achievement is often more closely tied to parental involvement relative to other subject areas. While the underlying cause of these findings has significant implications for constructing optimal education policy, this particular dataset does not lend itself to providing much insight into the root cause. Despite this, the good news is that schools take action and improve, even without the more immediate threats of private school voucher sanctions, when school performances are graded and published.

9

References Bush v. Holmes, Fl. SC04-2323, 4 (2006). Chakrabarti, R. (2007). Vouchers, public school response, and the roles of incentives: Evidence from Florida. (Staff Report No. 306) Retrieved from the Federal Bank of New York Staff Reports: http://www.newyorkfed.org/research/staff_reports/sr306.pdf Chiang, H. (2009). How accountability pressure on failing schools affects student achievement. Journal of Public Economics, 93, 1045-1057. Figlio, D.N., & Rouse, C.E. (2005). Do accountability and voucher threats improve lowperforming schools? (Working Paper No.11597) Retrieved from the National Bureau of Economic Research: http://www.nber.org/papers/w11597 Florida Department of Education. (2002). Schools graded 'F' for 2001-2002. Florida School Grades - School Accountability Reports. Retrieved from http://schoolgrades.fldoe.org/pdf/0102/fschools.pdf Florida Department of Education (2008). School grades 1999-2008. Retrieved from http://schoolgrades.fldoe.org/pdf/0708/School_Grades_08_PressPacket6_7.pdf Florida Department of Education. (2010). History of school grades. Florida School Grades School Accountability Reports. Retrieved from http://schoolgrades.fldoe.org/pdf/0809/SchoolGrades2009_11.pdf Florida Department of Education. (2010). OSP Scholarship Program: Opportunity Scholarship Program public school option. Florida Department of Education: Florida School Choice. Retrieved from http://www.floridaschoolchoice.org/Information/OSP/files/Fast_Facts_OSP.pdf Goldhaber, D., & Hannaway, J. (2004). Accountability with a kicker: Observations on the Florida A+ Accountability Plan. Phi Delta Kappan, April 2004, 598-605.

10

Greene, J.P. (2001). An evaluation of the Florida A-Plus Accountability and School Choice Program. Retrieved from http://www.hks.harvard.edu/var/ezp_site/storage/fckeditor/file/pdfs/centersprograms/centers/taubman/working_papers/greene_01_florida.pdf Greene, J.P., & Winters, M.A. (2003). When schools compete: The effects of vouchers on Florida public school achievement. (Education Working Paper No. 2). Retrieved from the Center for Civic Innovation at the Manhattan Institute: http://www.manhattaninstitute.org/html/ewp_02.htm Ladd, H.F., & Glennie, E.J. (2001). A replication of Jay Greene's voucher effect study using North Carolina data. In Carnoy, M. (Ed.), School vouchers: Examining the evidence (pp.49-52). Washington, D.C.: Economic Policy Institute. National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Retrieved from http://datacenter.spps.org/sites/2259653e-ffb3-45ba8fd6-04a024ecf7a4/uploads/SOTW_A_Nation_at_Risk_1983.pdf Rouse, C.E., Hannaway, J., Goldhaber, D., & Figlio, D.N. (2007). Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressure. (Working Paper No. 13681) Retrieved from the National Bureau of Economic Research: http://www.nber.org/papers/w13681

West, M.R., & Peterson, P.E. (2006). The efficacy of choice threats within school accountability systems: Results from legislatively induced experiments. The Economic Journal, 116, C46-C62.

11

Table 1 – Descriptive Statistics for 4th Grade in Schools Close to the D/F Cut Point Variable DSS_Math lag_Math DSS_Reading Lag_Reading

N 13281 13281 13235 13238

Mean 1405 1251 1412 1193

Std. Dev. 249 283 296 318

Min 581 375 295 86

Max 2330 2225 2527 2514

FRL Disability Male Minority

13281 13281 13281 13281

0.901 0.210 0.505 0.721

0.299 0.408 0.500 0.449

0 0 0 0

1 1 1 1

2003 2004 2006 2007 2008

13281 13281 13281 13281 13281

0.136 0.150 0.066 0.262 0.220

0.342 0.357 0.248 0.440 0.414

0 0 0 0 0

1 1 1 1 1

F Lag F*Threat

13281 13281

0.248 0.052

0.432 0.222

0 0

1 1

Note: DSS is the FCAT developmental scale score. FRL is a dichotomous variable for whether students have obtained free or reduced lunch status. Disability is a dichotomous variable for whether the student has been identified with a learning disability. For all regressions, the 2005-06 school year is the omitted year variable.

12

Table 2 - Descriptive Statistics for 5th Grade in Schools Close to the D/F Cut Point Variable DSS_Math lag_Math DSS_Reading Lag_Reading

N 13493 13493 13458 13441

Mean 1507 1352 1425 1363

Std. Dev. 251 256 312 320

Min 569 569 474 295

Max 2456 2330 2713 2638


13493 13493 13493 13493

0.899 0.210 0.498 0.732

0.301 0.407 0.500 0.443

0 0 0 0

1 1 1 1

2003 2004 2006 2007 2008

13493 13493 13493 13493 13493

0.178 0.125 0.062 0.260 0.215

0.382 0.331 0.241 0.439 0.411

0 0 0 0 0

1 1 1 1 1

F Lag F*Threat

13493 13493

0.249 0.063

0.433 0.244

0 0

1 1


13

Table 3 - Descriptive Statistics for 6th Grade in Schools Close to the D/F Cut Point Variable DSS_Math lag_Math DSS_Reading Lag_Reading

N 19796 19796 19693 19707

Mean 1499 1489 1472 1395

Std. Dev. 271 264 311 318

Min 770 375 539 295

Max 2492 2291 2758 2713


19796 19796 19796 19796

0.886 0.225 0.522 0.711

0.317 0.418 0.500 0.453

0 0 0 0

1 1 1 1

2003 2004 2006 2007 2008

19796 19796 19796 19796 19796

0.132 0.260 0.011 0.402 0.089

0.338 0.439 0.102 0.490 0.284

0 0 0 0 0

1 1 1 1 1

F Lag F*Threat

19796 19796

0.215 0.103

0.411 0.304

0 0

1 1


14

Table 4 – Stigma and Voucher Sanction Threat Impacts on Math Scores Variables (Tist-1) lag_Math lag_Math2 lag_Math3 Xit FRL Disability Male Minority Yt Y_2003 Y_2004 Y_2006 Y_2007 Y_2008

Fst-1 F*Threat

Constant Observations R-squared

(1) Grade 4

(2) Grade 5

(3) Grade 6

(4) Grade 4

(5) Grade 5

(6) Grade 6

-1.100*** [0.112] 0.00134*** [9.29e-05] -3.31e-07*** [2.53e-08]

-1.861*** [0.215] 0.00187*** [0.000164] -4.32e-07*** [4.09e-08]

-1.872*** [0.178] 0.00152*** [0.000131] -2.65e-07*** [3.05e-08]

-0.972*** [0.136] 0.00123*** [0.000119] -3.01e-07*** [3.40e-08]

-1.731*** [0.225] 0.00178*** [0.000174] -4.13e-07*** [4.39e-08]

-1.895*** [0.191] 0.00153*** [0.000142] -2.66e-07*** [3.32e-08]

-18.99*** [5.414] -55.52*** [4.726] -2.220 [3.004] -37.10*** [6.982]

-10.28* [5.610] -68.47*** [6.104] -6.797** [3.023] -30.53*** [5.114]

-24.93*** [5.041] -56.84*** [5.052] -12.48*** [2.712] -5.188 [6.797]

-21.68*** [5.987] -56.06*** [5.308] -0.537 [3.549] -43.31*** [7.637]

-7.803 [6.430] -63.62*** [6.522] -11.43*** [3.423] -35.93*** [5.861]

-27.56*** [5.036] -53.33*** [4.734] -12.23*** [2.976] -3.567 [7.065]

17.51 [13.54] 4.000 [13.87] 15.12 [14.74] 21.23** [8.818] 35.22*** [11.78]

-0.306 [14.39] -5.385 [13.09] 5.911 [13.50] -31.34*** [11.50] -11.14 [11.02]

-51.77*** [17.51] -13.85 [14.75] -22.50 [16.87] -21.17* [11.72] 22.55 [17.43]

16.55 [13.49] 2.590 [13.80] 15.70 [14.82] 22.00** [8.674]

-0.684 [14.41] -6.135 [13.03] 6.216 [13.46] -31.31*** [11.56]

-52.37*** [17.48] -14.17 [14.68] -22.77 [16.99] -20.90* [11.68]

25.82*** [8.798] -14.29 [21.60]

22.21** [9.060] -28.48 [23.64]

11.16 [9.449] -8.855 [19.24]

20.37** [8.495] -8.438 [21.91]

18.53* [10.71] -23.92 [24.14]

9.126 [9.289] -7.129 [19.19]

1,348*** [47.46] 13,281 0.543

1,720*** [93.37] 13,493 0.554

1,817*** [73.60] 19,796 0.585

1,312*** [54.50] 10,360 0.525

1,660*** [98.86] 10,596 0.544

1,833*** [77.21] 18,038 0.580

Note: Robust standard errors in brackets. *** - significant at p < 0.01, ** - significant at p < 0.05, * - significant at p < 0.10

15

Table 5 – Stigma and Voucher Sanction Threat Impacts on Reading Scores Variables (Tist-1) lag_Reading lag_Reading2 lag_Reading3 Xit FRL Disability Male Minority Yt Y_2003 Y_2004 Y_2006 Y_2007 Y_2008

Fst-1 F*Threat

Constant Observations R-squared

(1) Grade 4

(2) Grade 5

(3) Grade 6

(4) Grade 4

(5) Grade 5

(6) Grade 6

-0.308*** [0.0842] 0.000872*** [8.48e-05] -2.43e-07*** [2.74e-08]

-0.989*** [0.1000] 0.00120*** [9.03e-05] -2.59e-07*** [2.52e-08]

-1.480*** [0.0971] 0.00147*** [8.40e-05] -3.05e-07*** [2.21e-08]

-0.329*** [0.0956] 0.000891*** [9.99e-05] -2.48e-07*** [3.33e-08]

-0.900*** [0.117] 0.00112*** [0.000109] -2.36e-07*** [3.10e-08]

-1.502*** [0.111] 0.00148*** [9.52e-05] -3.05e-07*** [2.50e-08]

-17.10** [7.878] -67.99*** [5.394] -22.17*** [3.761] -50.42*** [6.888]

-22.34*** [5.771] -54.97*** [6.471] -23.43*** [3.976] -20.74*** [6.056]

-30.05*** [5.954] -61.48*** [5.641] -15.90*** [3.123] -11.04** [5.033]

-15.57* [9.189] -64.78*** [6.269] -19.13*** [4.291] -54.96*** [8.069]

-24.78*** [6.376] -46.10*** [6.941] -22.86*** [4.637] -22.97*** [7.078]

-32.19*** [5.761] -59.34*** [5.461] -16.42*** [3.331] -9.721* [5.267]

52.78*** [11.33] 56.08*** [17.58] -28.15* [15.09] 12.12 [9.459] 37.34*** [9.461]

-4.378 [11.24] 8.515 [14.30] 57.34*** [12.17] 26.35*** [8.485] 44.33*** [8.854]

-82.64*** [11.74] -57.11*** [10.78] -42.20*** [11.55] -67.23*** [9.852] -3.378 [13.10]

52.25*** [11.31] 55.37*** [17.60] -27.76* [15.16] 12.47 [9.349]

-4.470 [11.32] 8.564 [14.26] 57.76*** [12.17] 26.51*** [8.488]

-82.12*** [11.76] -56.66*** [10.75] -42.35*** [11.49] -67.43*** [9.842]

22.09*** [8.295] -30.36** [14.59]

12.27* [7.103] -35.52* [20.73]

4.559 [10.70] -20.40 [13.81]

18.13* [9.412] -26.19* [15.19]

11.18 [8.197] -34.35* [20.32]

4.396 [10.79] -20.35 [13.82]

995.8*** [31.99] 13,265 0.526

1,209*** [35.78] 13,482 0.587

1,596*** [37.12] 19,833 0.564

1,003*** [35.57] 10,348 0.514

1,183*** [38.93] 10,587 0.577

1,607*** [42.73] 18,081 0.561

Note: Robust standard errors in brackets. *** - significant at p < 0.01, ** - significant at p < 0.05, * - significant at p < 0.10

16